Testing statistical significance scores of sequence comparison methods with structure similarity

Area of Science:

• Bioinformatics
• Computational Biology
• Structural Bioinformatics

Background:

• The Smith-Waterman algorithm is popular for sequence comparison, but statistical significance testing impacts search quality.
• E-values are common, while Z-scores (Monte-Carlo based) are used in some databases.
• Previous studies suggested Z-scores outperform e-values on simulated data.

Purpose of the Study:

• To validate the superiority of Z-scores over e-values using real, evolutionarily related protein sequences.
• To compare the performance of Smith-Waterman implementations with different statistical tests.

Main Methods:

• Evaluation of Smith-Waterman algorithm with e-value and Z-score statistics on the ASTRAL SCOP database.
• Utilized ROC, CVE, and AP measures for performance assessment.
• BLAST and FASTA algorithms served as reference benchmarks.

Main Results:

• Two of three Smith-Waterman implementations using e-values demonstrated superior prediction of protein structural similarities compared to Z-score.
• The SSEARCH implementation, in particular, achieved very high performance scores.
• Smith-Waterman implementations generally outperformed heuristic methods like BLAST and FASTA.

Conclusions:

• The computationally intensive Z-score offers no clear advantage over the e-value for sequence comparison.
• Smith-Waterman algorithms provide better results than heuristic approaches.
• Recommends using the SSEARCH algorithm with e-values for pairwise sequence comparisons.