Benchmarking framework for machine learning classification from fNIRS data.

Area of Science:

• Neuroscience
• Machine Learning
• Biomedical Engineering

Background:

• Lack of standardized machine learning practices for functional near-infrared spectroscopy (fNIRS) data hinders reliable brain-computer interface (BCI) development.
• Inconsistent reporting and absence of open-source benchmarks make evaluating BCI model generalizability difficult.

Purpose of the Study:

• Establish a best-practice, open-source benchmarking framework (BenchNIRS) for evaluating machine learning models applied to fNIRS data.
• Provide a standardized methodology for optimizing and assessing BCI model performance using fNIRS.

Main Methods:

• Developed BenchNIRS, an open-source framework utilizing nested cross-validation on five public fNIRS datasets for BCI.
• Benchmarked six machine learning models (LDA, SVM, kNN, ANN, CNN, LSTM) against varying training data and time window sizes.
• Investigated performance differences between sliding window vs. epoch classification and personalized vs. generalized approaches.

Main Results:

• Model performance on unseen data was generally lower than reported in literature, with minimal differences between models.
• Benchmarking highlighted the persistent difficulty in achieving high generalizability for fNIRS-based BCIs.
• The study identified factors influencing classification performance, including data quantity and temporal features.

Conclusions:

• BenchNIRS offers a standardized tool for researchers to rigorously evaluate and compare machine learning models for fNIRS-based BCIs.
• The findings underscore the need for transparent reporting and robust validation in fNIRS BCI research.
• Recommendations are provided for methodology and reporting to advance the field of machine learning with fNIRS data.