A Bayesian method for comparing and combining binary classifiers in the absence of a gold standard

Area of Science:

• Bioinformatics
• Computational Biology
• Genomics

Background:

• Bioinformatics classification problems often rely on features like sequence and structure.
• Evaluating and combining multiple classifiers is crucial for improved performance.
• A gold standard dataset is not always available for classifier evaluation.

Purpose of the Study:

• To adapt a Bayesian model for comparing medical diagnostics to the bioinformatics domain.
• To develop an implementation applicable to any number of classifiers.
• To find the globally optimal logical combination of classifiers.

Main Methods:

• Applied a Bayesian model for classifier comparison without a gold standard.
• Implemented a novel approach for combining any number of classifiers.
• Tested the model on protein subcellular localization, swine flu diagnostics, and genome-wide association studies.

Main Results:

• The Bayesian model provided accurate sensitivity and specificity estimates, closely matching gold standard results.
• Classifiers were correctly ranked, and run times were feasible for large datasets.
• The optimal logical combination for all tested datasets was the union of classifiers.

Conclusions:

• The Bayesian method is suitable for diverse bioinformatics classification tasks, from small to genome-wide scales.
• The approach is robust to dependencies between classifiers.
• The union of classifiers is proposed as a generally optimal combination strategy.