Frequency-Aware Domain Generalization.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Deep Neural Networks (DNNs) display remarkable zero-shot generalization and emergent behaviors.
• The mechanisms driving these DNN capabilities, particularly generalization, are not fully understood.
• Understanding how DNNs perceive image frequencies is key to explaining their generalization.

Purpose of the Study:

• To investigate the link between DNN generalization and image frequency perception.
• To enhance DNN generalization performance by improving frequency awareness.
• To develop novel methods for boosting frequency awareness in DNNs.

Main Methods:

• Analyzing DNNs' perception of image frequencies to identify frequency-aware regions.
• Implementing frequency decomposition and mixup to learn high-frequency component-label relations.
• Utilizing hierarchical feature alignment for submodel guidance on frequency awareness.
• Ensembling submodels during inference to extract multi-frequency band features.

Main Results:

• A strong association was found between wider frequency-aware regions and enhanced DNN generalization.
• The proposed methods successfully broadened DNNs' frequency awareness.
• Significant improvements in generalization performance were observed in image classification and object detection.
• The approach demonstrated effectiveness in single and multi-source domain generalization.

Conclusions:

• DNN generalization is intrinsically linked to their ability to process image frequencies.
• Broadening frequency awareness is an effective strategy for improving DNN generalization.
• The proposed methods are scalable, plug-and-play, and applicable across various DNN architectures and tasks.