They watched two separate sets of Hollywood movie trailers, while fMRI was used to measure blood flow through the visual cortex, the part of the brain that processes visual information. On the computer, the brain was divided into small, three-dimensional cubes known as volumetric pixels, or “voxels.” The brain activity recorded while subjects viewed the first set of clips was fed into a computer program that learned, second by second, to associate visual patterns in the movie with the corresponding brain activity. Brain activity evoked by the second set of clips was used to test the movie reconstruction algorithm. This was done by feeding 18 million seconds of random YouTube videos into the computer program so that it could predict the brain activity that each film clip would most likely evoke in each subject. Finally, the 100 clips that the computer program decided were most similar to the clip that the subject had probably seen were merged to produce a blurry yet continuous reconstruction of the original movie.And here’s the astonishing result:
[youtube http://www.youtube.com/watch?v=nsjDnYxJ0bo&w=560&h=315]
While any practical application is decades away, it’s easy to imagine how such technology could be used to create new tech tools to communicate with or at least understand what’s going on inside the minds of comatose or non-communicative people or to create new ways for disabled people to control computers with their minds. And the rest of us will have the chance to watch our dreams—and nightmares—over and over again.
