Neural network learns to reproduce what your brain sees

Scientists dream of recreating mental images through brain scans, but current techniques produce results that are... fuzzy, to put it mildly. A trio of Chinese researchers might just solve that. They've developed neural network algorithms that do a much better job of reproducing images taken from functional MRI scans. The team trains its network to recreate images by feeding it the visual cortex scans of someone looking at a picture and asking the network to recreate the original image based on that data. After enough practice, it's off to the races -- the system knows how to correlate voxels (3D pixels) in scans so that it can generate accurate, noise-free images without having to see the original.

You can see the results for yourself in the comparison above. The new technique, DGMM (Deep Generative Multiview Model), is very nearly on par with the source material. Virtually all the other approaches are so imprecise and noisy that many of their results are either difficult to read or just plain wrong.

This initial foray revolved around people staring at simple images. There's much more work to be done before it's clear this method works for complex images and videos -- you know, real life. Still, the breakthrough hints at a bright (if slightly creepy) future for brain image recreation. It could help with brain-controlled devices, for example, or help doctors understand mental health issues based on dreams or hallucinations.