A new framework of design and continuous evaluation to improve brain training

Area of Science:

• Cognitive Science
• Educational Psychology
• Neuroscience

Background:

• Effective teaching hinges on accurate evaluation to understand abilities and set learning outcomes.
• Methods for effective evaluation are often lacking across disciplines, hindering the assessment of training efficacy.
• The field of brain training faces significant criticism, partly due to challenges in evaluation.

Purpose of the Study:

• To discuss issues in effective evaluation within the context of brain training.
• To identify common problems in test validity, reliability, teaching to the test, expectation effects, and statistical rigor.
• To propose a new paradigm synthesizing evaluation and training.

Main Methods:

• Review of common issues in test validity and reliability across various fields, including brain training.
• Analysis of commonalities between evaluation goals and adaptive training procedures.
• Proposal of a continuous evaluation paradigm.

Main Results:

• Problems in brain training evaluation mirror those in other fields, including poor test validity and reliability.
• Adaptive training procedures share commonalities with evaluation goals.
• Continuous evaluation, integrating testing into training, offers a potential solution.

Conclusions:

• Continuous evaluation may enhance skill evaluation reliability through longitudinal sampling.
• Integrating testing into training can improve validity by better aligning evaluation with learning objectives.
• This synthesized approach offers a path towards more effective brain training and skill assessment.