Visual Semantic Encoding and Identification of Naturalistic Movies via High-Density Diffuse Optical Tomography.

Area of Science:

• Neuroscience
• Cognitive Science
• Biomedical Engineering

Background:

• Understanding brain's semantic representation is crucial for neuroscience and clinical applications.
• Existing brain encoding/decoding models use constrained environments or limited conventional functional near-infrared spectroscopy (fNIRS).
• Conventional fNIRS is limited by sparse sampling and block-design paradigms.

Purpose of the Study:

• To investigate the efficacy of high-density diffuse optical tomography (HD-DOT) for semantic encoding and decoding using naturalistic movie stimuli.
• To assess HD-DOT's capability in capturing high-dimensional semantic representations in the brain.
• To bridge the gap between fMRI-level semantic mapping and the accessibility of optical imaging.

Main Methods:

• Collected 3.5 hours of naturalistic movie viewing data from six participants.
• Utilized high-density diffuse optical tomography (HD-DOT) for advanced tomographic optical imaging.
• Developed encoding models to predict voxel-level brain responses and performed decoding analyses to identify viewed clips.

Main Results:

• Encoding models robustly predicted voxel-level responses, generating semantic category maps comparable to fMRI studies.
• Decoding analyses demonstrated that HD-DOT responses contained sufficient semantic information to identify viewed movie clips.
• Identified a shared low-dimensional semantic space across participants and revealed interpretable semantic dimensions (e.g., social agents, objects, scenes).

Conclusions:

• HD-DOT can effectively recover distributed, high-dimensional semantic representations from naturalistic movie stimuli.
• This study validates HD-DOT as a viable tool for semantic brain mapping, offering advantages over conventional methods.
• Findings pave the way for more accessible and ecologically valid brain imaging studies of semantic processing.