Efforts to enhance reproducibility in a human performance research project

Area of Science:

• Biological research
• Human performance studies
• Defense research

Background:

• Ensuring research validity is crucial for funding agencies.
• Open science promotes reproducibility through data and code sharing.
• Sensitive research may require alternative validation methods beyond broad data distribution.

Purpose of the Study:

• To describe an independent verification and validation (IV&V) strategy for a biological research program.
• To promote reproducibility in research where data sharing is restricted.
• To enhance the quality and validity of scientific output.

Main Methods:

• Implemented a four-pronged IV&V strategy in the Measuring Biological Aptitude program.
• Established a centralized data storage and exchange platform.
• Conducted quality assurance/quality control (QA/QC) of data collection, model testing, and data archival.

Main Results:

• The IV&V plan was successfully executed with agency and researcher collaboration.
• QA/QC improved data collection processes and identified experimental errors.
• Holdout validation confirmed the performance of research models.

Conclusions:

• Independent validation teams can ensure reproducibility in sensitive research contexts.
• This targeted approach supports scientific criticism and enhances research quality.
• IV&V is a valuable tool for promoting scientific integrity when open data sharing is not feasible.