Confidence Estimation via Auxiliary Models.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Quantifying deep neural classifier confidence is crucial for safety-critical applications.
• Standard maximum class probability (MCP) has limitations for accurate confidence estimation.

Purpose of the Study:

• Introduce a novel confidence estimation criterion: true class probability (TCP).
• Develop a method to learn TCP from data using an auxiliary model.
• Evaluate the proposed approach in failure prediction and domain adaptation tasks.

Main Methods:

• Proposed a new target criterion for model confidence: true class probability (TCP).
• Developed a specific learning scheme to learn TCP from data with an auxiliary model.
• Evaluated the approach on image classification and semantic segmentation tasks.

Main Results:

• True class probability (TCP) demonstrates superior properties for confidence estimation compared to MCP.
• The proposed approach significantly outperforms strong baselines across various benchmarks.
• Effective confidence estimates were achieved in failure prediction and domain adaptation.

Conclusions:

• The novel TCP criterion and learning scheme provide a more reliable method for quantifying deep learning model confidence.
• This approach enhances the deployment of deep learning models in safety-critical domains.
• The method shows broad applicability across different network architectures and dataset sizes.