Sharpness-Aware Fine-Tuning for OOD Detection

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computer Science

Background:

• Out-of-distribution (OOD) detection is vital for reliable machine learning deployment.
• Traditional methods often struggle with distinguishing in-distribution (ID) from OOD data.
• Sharpness-aware Minimization (SAM) shows promise in improving model generalization.

Purpose of the Study:

• To investigate the efficacy of Sharpness-aware Minimization (SAM) for OOD detection.
• To propose a SAM-based fine-tuning approach to enhance OOD detection performance.
• To develop a time-efficient fine-tuning strategy using a specialized loss function.

Main Methods:

• Fine-tuning machine learning models using Sharpness-aware Minimization (SAM) instead of standard optimizers like SGD.
• Utilizing a carefully designed loss function to facilitate efficient fine-tuning.
• Evaluating the method's impact on the separation of in-distribution (ID) and out-of-distribution (OOD) data score distributions.

Main Results:

• SAM-based fine-tuning significantly improves OOD detection performance.
• The method effectively pushes score distributions of ID and OOD data further apart.
• Performance gains are achieved rapidly, often within a single epoch of fine-tuning.
• The approach demonstrates flexibility, enhancing various existing OOD detection techniques.
• State-of-the-art results are achieved on standard OOD benchmarks across diverse architectures.

Conclusions:

• Sharpness-aware Minimization (SAM) offers a powerful new perspective for improving out-of-distribution (OOD) detection.
• The proposed SAM-based fine-tuning method is efficient and highly effective.
• This approach provides a flexible and robust strategy for enhancing the reliability of machine learning models in real-world scenarios.