Evaluating virtual screening methods: good and bad metrics for the "early recognition" problem.

Area of Science:

• Computational Chemistry
• Bioinformatics
• Drug Discovery

Background:

• Virtual screening (VS) relies on ranking methods to identify potential drug candidates.
• Existing metrics like ROC, AUAC, and EF have limitations in evaluating early recognition performance.

Purpose of the Study:

• To identify flaws in current VS ranking metrics, particularly the 'early recognition problem'.
• To propose a novel, improved metric for VS performance evaluation.

Main Methods:

• Mathematical derivation of relationships between existing metrics (ROC, AUAC, average rank).
• Formal generalization of the ROC metric for early recognition.
• Development of the Boltzmann-enhanced discrimination of receiver operating characteristic metric.

Main Results:

• ROC, AUAC, and average rank metrics exhibit inappropriate behavior for early recognition.
• The EF metric lacks sensitivity to ranking performance around a cutoff.
• The proposed Boltzmann-enhanced discrimination metric combines RIE's discrimination power with ROC's statistical significance.

Conclusions:

• Standard VS metrics are inadequate for comparing methods prioritizing early identification of actives.
• The novel Boltzmann-enhanced discrimination metric offers a more robust evaluation.
• Recommendations are provided for optimizing VS metric parameters and understanding error sources.