Beyond p-values in the evaluation of brain-computer interfaces: A Bayesian estimation approach.

Area of Science:

• Neuroscience
• Computer Science
• Statistics

Background:

• Null hypothesis significance testing (NHST) using p-values is standard for evaluating brain-computer interface (BCI) performance.
• Over-reliance on NHST contributes to the reproducibility crisis in neuroscience and psychology.

Purpose of the Study:

• Propose Bayesian estimation as a superior alternative to NHST for BCI performance data analysis.
• Introduce hierarchical generalized linear models (HGLMs) as a flexible framework for BCI data.

Main Methods:

• Developed hierarchical Bayesian models for common BCI experimental designs (t-test, linear regression, ANOVA).
• Demonstrated that these models are special cases of the proposed HGLM framework.
• Applied Bayesian inference for parameter estimation.

Main Results:

• Effectively demonstrated the proposed Bayesian models on three real-world BCI datasets.
• Showcased how Bayesian estimation provides nuanced insights into BCI performance.
• Provided open-source data and code for result reproducibility.

Conclusions:

• Bayesian estimation with HGLMs offers greater flexibility and straightforward interpretation for nested BCI experimental designs compared to NHST.
• Wider adoption of Bayesian methods can increase transparency, facilitate knowledge accumulation, and mitigate questionable research practices like 'p-hacking'.