Reliable prediction intervals with regression neural networks.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• Conventional regression neural networks (NNs) provide point predictions, lacking reliable confidence measures.
• Accurate confidence intervals are crucial for decision-making in various scientific and engineering applications.
• Existing methods often require strong assumptions about data distribution.

Purpose of the Study:

• To extend conventional NNs with prediction intervals that guarantee a specified level of confidence.
• To introduce a novel machine learning framework, Conformal Prediction (CP), for reliable uncertainty quantification.
• To evaluate the proposed method on benchmark datasets and a real-world application (Total Electron Content prediction).

Main Methods:

• Implementation of Conformal Prediction (CP) as a post-processing step for neural network regression.
• Utilizing the assumption of independent and identically distributed (i.i.d.) data for theoretical guarantees.
• Empirical evaluation on four benchmark datasets and a large-scale Total Electron Content (TEC) dataset spanning 11 years.

Main Results:

• The proposed method successfully generates prediction intervals with guaranteed confidence levels.
• Experimental results demonstrate that the prediction intervals are well-calibrated.
• The generated intervals are sufficiently narrow (tight) for practical utility in applications like TEC prediction.

Conclusions:

• Conformal Prediction offers a robust framework for uncertainty quantification in neural network regression.
• The method provides reliable and practically useful prediction intervals without strong distributional assumptions.
• This approach enhances the trustworthiness and applicability of NNs in scientific domains requiring confidence measures.