Bi-convex Optimization to Learn Classifiers from Multiple Biomedical Annotations.

Area of Science:

• Machine Learning
• Computational Biology
• Data Science

Background:

• Classifiers trained on data from multiple annotators often face challenges due to inconsistent labels.
• Existing methods primarily focus on modeling annotator behavior or using consensus, with many employing logistic regression loss.

Purpose of the Study:

• To extend existing probabilistic algorithms for classifier construction to utilize hinge loss, commonly used in support vector machines.
• To develop methods that simultaneously construct classifiers and estimate the reliability of each annotator.

Main Methods:

• Formulated bi-convex programs to integrate classifier construction and annotator reliability estimation.
• Modified hinge loss by incorporating a weighted combination of annotator labels, with weights determined by annotator reliability.
• Developed efficient alternating algorithms to solve the proposed bi-convex programs.

Main Results:

• Proposed methods demonstrated strong performance, either outperforming or being competitive with state-of-the-art approaches.
• Evaluated on benchmark datasets and three real-world biomedical problems, confirming the effectiveness of the approach.
• Annotator reliability parameters were shown to be adaptable (constant, class-dependent, or example-dependent).

Conclusions:

• The developed bi-convex programming approach offers a robust framework for constructing classifiers from noisy, multi-annotator data.
• Simultaneous estimation of classifier and annotator reliability provides a more accurate and adaptable model.
• The methods are applicable to various domains, including critical biomedical applications, offering improved performance over existing techniques.