FastPval: a fast and memory efficient program to calculate very low P-values from empirical distribution

Area of Science:

• Computational Biology
• Statistical Computing

Background:

• Resampling methods like permutation and bootstrap are crucial for statistical significance testing.
• Calculating very low P-values necessitates large resampling sizes, leading to computational bottlenecks.

Purpose of the Study:

• To develop an efficient program for calculating very low P-values from large resampled datasets.
• To overcome the computational limitations of traditional resampling methods.

Main Methods:

• Developed FastPval, a multi-stage P-value calculation program.
• Utilized an efficient algorithm for P-value computation from empirical distributions.

Main Results:

• FastPval calculates P-values up to 10^-9 efficiently, even on personal computers.
• Achieved significant reductions in computation time (52.94%) and resource consumption (0.11% memory/storage) compared to conventional methods.
• Successfully processed datasets too large for traditional methods, with accuracy comparable to exact ranking.

Conclusions:

• FastPval offers a highly efficient and resource-sparing solution for calculating low P-values.
• The tool is accessible and applicable to large-scale biological and statistical analyses.