Entropic Out-of-Distribution Detection: Seamless Detection of Unknown Examples.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Deep Learning

Background:

• Neural networks exhibit poor out-of-distribution (OOD) detection performance.
• This is primarily attributed to SoftMax loss anisotropy and low-entropy probability distributions.
• Existing OOD detection methods often circumvent SoftMax limitations, leading to side effects like reduced accuracy and increased complexity.

Purpose of the Study:

• To address the limitations of SoftMax loss in OOD detection.
• To introduce a novel loss function, IsoMax loss, as a direct replacement for SoftMax loss.
• To improve OOD detection performance without compromising classification accuracy or inference efficiency.

Main Methods:

• Proposed IsoMax loss function, which is isotropic and promotes high-entropy posterior probability distributions.
• Replaced SoftMax loss with IsoMax loss in neural network training.
• Evaluated IsoMax loss as a drop-in replacement for SoftMax loss.

Main Results:

• IsoMax loss effectively improves neural network OOD detection performance.
• Models trained with IsoMax loss maintain classification accuracy and inference speed comparable to SoftMax loss.
• The method does not require additional data, hyperparameters, or complex techniques.

Conclusions:

• IsoMax loss offers a seamless and effective solution for enhancing OOD detection in neural networks.
• It serves as a robust baseline for OOD detection, combinable with other methods for superior results.
• This approach overcomes SoftMax loss drawbacks without introducing negative side effects.