Simple and statistically sound recommendations for analysing physical theories

Area of Science:

• Physics
• Astrophysics
• Cosmology
• Statistical Inference

Background:

• Physical theories with numerous parameters or tested across multiple experiments present significant statistical inference challenges.
• Commonly used ad hoc methods for parameter estimation are statistically unsound and problematic in high dimensions.
• Particle physics, astrophysics, and cosmology frequently encounter these complex inference scenarios.

Purpose of the Study:

• To provide statistically sound methods for inference in complex physical theories.
• To guide physicists lacking extensive statistical training towards correct scientific conclusions.
• To recommend accessible software tools and standards for robust data analysis.

Main Methods:

• Guidance on statistically sound inference procedures beyond ad hoc methods.
• Recommendations for readily available software tools and standards.
• Illustrative examples reproducible with publicly available code.

Main Results:

• Demonstration of pathologies in common ad hoc statistical methods.
• Presentation of clear guidance for improved statistical inference.
• Identification of practical tools and standards for physicists.

Conclusions:

• Adopting recommended methods ensures more accurate scientific conclusions in physics.
• Statistically sound inference is achievable with a modest increase in analysis burden.
• Publicly available code enables reproducibility and validation of findings.