Mind the Noise When Identifying Computational Models of Cognition from Brain Activity

Area of Science:

• Computational Neuroscience
• Cognitive Neuroscience
• Biophysics

Background:

• Modeling studies in computational neuroscience are vital for understanding brain function.
• The validity of these models can be compromised by measurement error and data limitations.
• Assessing model accuracy requires rigorous validation techniques.

Purpose of the Study:

• To investigate the impact of measurement error on the validity of computational modeling studies.
• To evaluate how data quality and quantity influence model selection in neuroscience.
• To propose methods for ensuring the reliability of computational models.

Main Methods:

• Developed a synthetic validity test using simulated P300 event-related potentials.
• Employed four computational models of P300 amplitude fluctuations with varying complexity and dependency.
• Utilized Bayesian model selection based on exceedance probabilities under different error levels and data points.

Main Results:

• The simplest model was often favored over the data-generating model with insufficient data.
• Model misidentification occurred with low data quality and quantity, particularly when model predictors were correlated.
• Sufficient data quality and quantity enabled correct identification of the data-generating model, even against similar models.

Conclusions:

• Data quality and the number of data points are critical factors affecting computational model validity.
• Model space characteristics, including complexity and dependency, must be considered.
• Synthetic validity tests and mandatory simulations are recommended to ensure robust computational modeling.