Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Test-time adaptation (TTA) enhances model performance on unseen data by adapting during testing.
• Existing TTA methods struggle with unstable online model updates, particularly with mixed shifts, small batches, and imbalanced data.
• Batch normalization layers are identified as a key factor contributing to TTA instability.

Purpose of the Study:

• Investigate the causes of TTA instability.
• Develop novel methods to stabilize TTA and prevent model collapse.
• Improve the robustness and reliability of TTA for real-world applications.

Main Methods:

• Analyzed the impact of batch-agnostic normalization layers (group/layer norm) on TTA stability.
• Proposed SAR (sharpness-aware and reliable entropy minimization) to mitigate noisy samples and promote flat minima.
• Introduced SAR2, building on SAR, with redundancy and inequity regularizers to prevent representation collapse.

Main Results:

• TTA with group/layer norm shows improved stability but can still collapse.
• Model collapse is linked to gradient explosion/degradation and biased representations.
• SAR and SAR2 effectively stabilize TTA, outperforming prior methods in challenging scenarios.

Conclusions:

• Batch-norm-free TTA is more stable but prone to collapse.
• SAR and SAR2 offer robust solutions to TTA instability and representation collapse.
• The proposed methods are computationally efficient and suitable for real-world deployment.