Evaluating and Calibrating Uncertainty Prediction in Regression Tasks.

Area of Science:

• Machine Learning
• Computer Vision
• Data Science

Background:

• Accurate uncertainty prediction is crucial for safety-critical machine learning applications, especially in regression tasks.
• Existing definitions for regression uncertainty calibration have limitations in distinguishing informative from non-informative predictions.

Purpose of the Study:

• To address limitations in current regression uncertainty calibration definitions.
• To propose a novel definition and evaluation method for regression uncertainty calibration.
• To introduce a simple, effective calibration technique for regression tasks.

Main Methods:

• Developed a new definition for regression uncertainty calibration.
• Proposed a histogram-based evaluation method to cluster examples by uncertainty.
• Introduced a simple, scaling-based calibration method.

Main Results:

• The new definition effectively distinguishes informative from non-informative uncertainty predictions.
• The histogram-based evaluation method provides a robust way to assess calibration.
• The proposed scaling-based calibration method achieves performance comparable to more complex techniques.
• Validated on synthetic data and object detection bounding-box regression (COCO, KITTI datasets).

Conclusions:

• The proposed definition and evaluation method offer significant improvements for regression uncertainty calibration.
• A simple scaling-based calibration method is effective and practical for real-world applications.
• This work enhances the reliability of machine learning models in safety-critical domains.