Distilling the Unknown to Unveil Certainty.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Out-of-distribution (OOD) detection is crucial for AI systems to identify data samples that differ from the training data, ensuring reliable performance.
• Existing methods often struggle to robustly distinguish between in-distribution (ID) and OOD samples, particularly without access to ID training data.

Purpose of the Study:

• To develop a flexible framework for OOD knowledge distillation to improve OOD detection capabilities.
• To create a binary classifier that can reliably differentiate between ID and OOD samples under various conditions.

Main Methods:

• Introduction of Confidence Amendment (CA), a novel methodology to transform OOD samples into ID ones.
• Progressively amending prediction confidence from the network to enhance OOD sensitivity.
• Simultaneous synthesis of ID and OOD samples with adjusted confidence for classifier training.

Main Results:

• Theoretical analysis provided generalization error bounds for the binary classifier, highlighting the importance of confidence amendment.
• Extensive experiments demonstrated the proposed method's efficacy across diverse datasets and network architectures.
• The CA method successfully enhanced OOD sensitivity, leading to improved detection rates.

Conclusions:

• The proposed OOD knowledge distillation framework with Confidence Amendment is effective for robust OOD detection.
• The method offers a flexible approach applicable in scenarios with or without access to ID training data.
• Confidence Amendment plays a pivotal role in improving the sensitivity and reliability of OOD detection systems.