Related Experiment Video
Updated: Mar 15, 2026
08:36
The Immersive Cleveland Clinic Virtual Reality Shopping Platform for the Assessment of Instrumental Activities of Daily Living
Published on: July 28, 2022
4.6K
Dual modal data-driven virtual reality-based mild cognitive Impairment assessment using MCIformer.
Yanjie Zhang1, Yang Pan2, Shanshan Feng3
1Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, 999077, Hong Kong Special Administrative Region.
Computer Methods and Programs in Biomedicine
|March 13, 2026
Summary
This study introduces a novel Virtual Reality (VR) assessment for early detection of Mild Cognitive Impairment (MCI) by combining movement data and brain activity. The dual-modal approach significantly improves diagnostic accuracy for cognitive decline.
Area of Science:
- Neuroscience
- Medical Technology
- Artificial Intelligence
Background:
- Mild Cognitive Impairment (MCI) assessment is crucial for early intervention against Alzheimer's disease (AD).
- Virtual Reality (VR) offers engaging and ecologically valid cognitive assessments.
- Existing VR methods often miss subtle motor and neural indicators of cognitive decline.
Purpose of the Study:
- To develop a dual-modal, data-driven VR assessment integrating kinematic and functional near-infrared spectroscopy (fNIRS) data for MCI detection.
- To capture subtle motor deficits and neural connectivity changes indicative of early cognitive impairment.
- To enhance the accuracy and comprehensiveness of MCI assessment tools.
Main Methods:
- Developed a VR system collecting synchronized kinematic and fNIRS data from healthy and MCI participants.
- Extracted kinematic features (smoothness, coordination, stability) from movement trajectories.
- Analyzed fNIRS data to represent functional brain networks and interregional connectivity.
- Proposed MCIformer, a dual-modal fusion model using Transformers for kinematic sequences and Graph Transformers for fNIRS networks.
Main Results:
- The dual-modal system achieved 90% accuracy in MCI classification.
- This significantly outperformed models using only kinematic data (80%) or fNIRS data (85%).
- Integration of motor patterns and brain connectivity enhances classification by providing complementary information.
Conclusions:
- The VR-based dual-modal approach shows potential for accurate, scalable early MCI diagnosis in community settings.
- This method supports the development of advanced brain-behavior monitoring systems for cognitive health.
- The findings highlight the value of integrating diverse data modalities for robust cognitive assessment.
