Neurons++: A Unified Visual-Stream Framework for Image and Video Reconstruction from fMRI with Neural-to-Visual Interpretability

Key points

• Decoding visual experience from fMRI is important for understanding human perception and developing brain-computer interfaces.1
• Recent generative decoding methods can reconstruct plausible images and videos, but visual plausibility alone does not ensure that the output is faithfully grounded in brain activity.1
• Compared with our conference version Neurons, Neurons++ extends the paradigm from video-only reconstruction to visual-stream reconstruction for both images and videos, and replaces parallel task conditioning with serial cortex-inspired decoding.1
• Beyond reconstruction fidelity, neural-to-visual analyses indicate that Neurons++ exhibits qualitative directional controllability under neural perturbation, that its fMRI encoder read-out is sensitive to stimulus-relevant cortical features, and that it supports ROI-guided semantic modulation.1

Sources (1)

• [1]Neurons++: A Unified Visual-Stream Framework for Image and Video Reconstruction from fMRI with Neural-to-Visual Interpretability
  arXiv (AI, ML, NLP, CV, robotics, multi-agent) · Oct 9, 03:49 PM

  In this work, we propose Neurons++, a visual-stream fMRI reconstruction framework for both static image and dynamic video reconstruction.

  Decoding visual experience from fMRI is important for understanding human perception and developing brain-computer interfaces.

Extractive summary: sentences quoted from the sources.