Researchers at the Weizmann Institute of Science in Rehovot, Israel, have developed an AI tool capable of reconstructing visual images from human brain scans. The system utilizes functional magnetic resonance imaging (fMRI) data to predict both the structure and content of what a person is observing.
The training process involved eight volunteers who were shown approximately 9,000 images while inside high-resolution fMRI scanners. To overcome data limitations, the team developed an encoder-decoder framework where the encoder predicts brain activity from images, allowing the model to learn from datasets not originally paired with fMRI scans.
Approximately 70% of the training data consisted of images that were not initially associated with specific fMRI brain activity. The researchers noted that the model requires about 40 hours of fMRI data for initial calibration per subject, while the decoder itself can be trained with just one hour of data.
The technology can identify specific brain regions associated with categories such as food or sports. The findings were presented last month at the Cognitive Computational Neuroscience conference in New York. Future research aims to extend these capabilities to reconstruct video, audio, and human thoughts or dreams.
