The Fermi-Dirac distribution provides a calibrated probabilistic output for binary classifiers.

Area of Science:

• Machine Learning
• Statistical Physics
• Quantum Mechanics

Background:

• Binary classification is a fundamental machine learning task.
• Understanding its statistical properties is crucial for developing robust algorithms.
• Existing methods may lack calibrated probabilistic outputs.

Purpose of the Study:

• To explore the statistical properties of binary classification.
• To establish a formal relationship between binary classification and quantum physics.
• To develop a novel method for calibrated probabilistic outputs and ensemble learning.

Main Methods:

• Investigated the ranking statistics of binary classification problems.
• Drew parallels between classification probabilities and the Fermi-Dirac distribution.
• Derived a closed-form expression for the variance of the Area Under the Curve (AUC).
• Introduced FiDEL (Fermi-Dirac-based ensemble learning).

Main Results:

• A surprising equivalence was found between binary classification probabilities and the Fermi-Dirac distribution.
• The Area Under the Curve (AUC) is related to the temperature of an equivalent physical system.
• The optimal decision threshold corresponds to the chemical potential of the equivalent system.
• A closed-form expression for AUC variance was derived.

Conclusions:

• The Fermi-Dirac distribution provides a powerful framework for understanding and improving binary classifiers.
• This quantum-inspired approach enables calibrated probabilistic outputs and enhanced ensemble learning.
• FiDEL offers a novel algorithm for combining diverse classifiers effectively.