Dim(N) Week 8 - Nooroo Bae (22/08/24)
Neural decoding of mental imagery
[Paper]
Naoko K.-M., Shinji N., Kei M. (2024) Mental image reconstruction from human brain activity: Neural decoding of mental imagery via deep neural network-based Bayesian estimation, Neural Networks, 170, 349-363. https://doi.org/10.1016/j.neunet.2023.11.024.
[Abstract]
In this study, we have presented a machine learning method for visualizing subjective images in the human mind based on fMRI brain activity. While numerous previous studies provided machine learning methods to reconstruct visual stimuli from brain activity, the visualization of mental imagery had been left as a significant challenge. We enhanced one of the previous visual image reconstruction methods through the integration of Bayesian estimation and semantic assistance. The results not only highlight the efficacy of our proposed framework but also suggest its potential as a unique tool for directly delving into the subjective contents of the brain, encompassing phenomena such as illusions, hallucinations, and dreams.
[Summary]
Builds on Shen et al. (2019) by enhancing image reconstruction from brain activity.
Uses a Bayesian framework with DNNs (e.g., VGG19) and CLIP to decode brain signals.
Incorporates an image prior from VQGAN and uses the SGLD algorithm for sampling.
Successfully reconstructs seen and imagined images with improved accuracy and quality.
Comments