Low-Dimensional Gradient Helps Out-of-Distribution Detection.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Reliable deep neural network (DNN) deployment requires effective out-of-distribution (OOD) sample detection.
• Existing OOD detection methods primarily analyze forward pass information, overlooking backward pass gradient discrepancies.
• Current gradient-based methods often focus on gradient norms, neglecting valuable directional information.

Purpose of the Study:

• To investigate the utility of comprehensive gradient information, including directions, for OOD detection in DNNs.
• To address the challenge of high-dimensional gradient data for OOD detection.
• To develop an effective method for OOD detection by leveraging gradient information.

Main Methods:

• Proposed a novel approach utilizing the entirety of gradient information for OOD detection.
• Implemented linear dimension reduction on gradients using principal component analysis (PCA) to handle high dimensionality.
• Integrated the reduced gradient representation with existing OOD detection score functions.

Main Results:

• The proposed method demonstrates superior performance across various OOD detection tasks.
• Achieved an average 11.15% reduction in false positive rate at 95% recall (FPR95) on the ImageNet benchmark using ResNet50.
• Outperformed current state-of-the-art methods in OOD detection accuracy.

Conclusions:

• Leveraging the full spectrum of gradient information, particularly directions, enhances OOD detection capabilities.
• Dimension reduction techniques like PCA are effective in managing high-dimensional gradient data for OOD detection.
• The proposed gradient-based OOD detection method offers a promising direction for improving DNN reliability.