How to Tell the Truth with Statistics: The Case for Accountable Data Analyses in Team-based Science

Area of Science:

• Data science
• Scientific methodology
• Biostatistics

Background:

• Data analysis is crucial for scientific research but often lacks sufficient documentation, leading to errors and bias.
• Current standards for reproducibility and reporting have improved but do not fully capture the analytical process.
• The scientific process can benefit from the accountability principles seen in clinical practice.

Purpose of the Study:

• To propose a novel framework, the Accountable Data Analysis Process (ADAP), for comprehensive documentation of the data analysis lifecycle.
• To enhance transparency, reproducibility, and accountability in research data analysis.
• To facilitate supervised and transparent team participation in data analysis.

Main Methods:

• Developed the Accountable Data Analysis Process (ADAP) framework.
• Modeled ADAP on electronic health records, treating datasets as "patients" and actions as recorded events.
• Utilized a project management system to log all data-related activities.

Main Results:

• ADAP provides a structured method for recording statistical methodologies and team communications.
• The framework enables supervised and transparent collaboration among research team members.
• Implementation in academic health centers is discussed, highlighting benefits and challenges.

Conclusions:

• The Accountable Data Analysis Process (ADAP) offers a novel solution to improve data analysis integrity.
• ADAP promotes accountability and transparency, essential for team-based science and reproducible research.
• This framework has the potential to mitigate errors, bias, and fraud in scientific data analysis.