Language-Driven Spatial-Semantic Cross-Attention for Face Attribute Recognition With Limited Labeled Data.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Face attribute recognition (FAR) typically requires large-scale labeled datasets, limiting its practical application.
• Existing methods often necessitate extensive pretraining on external datasets or complex auxiliary tasks.
• There is a need for efficient FAR methods that perform well with limited labeled data.

Purpose of the Study:

• To propose a novel method, language-driven spatial-semantic cross-attention (LSA), for face attribute recognition (FAR).
• To enhance FAR performance without requiring pretraining on additional datasets or auxiliary tasks.
• To leverage language-based relational information to improve attribute recognition accuracy.

Main Methods:

• Developed a language-driven spatial-semantic cross-attention (LSA) mechanism.
• Integrated language-driven knowledge with learned scaled-dot product attention.
• Introduced a correlation dictionary based on text embedding similarity between facial attributes and regions to represent relationships.
• Combined this correlation dictionary into a cross-attention framework with balancing parameters.

Main Results:

• The proposed LSA method achieved state-of-the-art performance on benchmark datasets (CelebA and LFWA).
• Demonstrated significant improvements with limited labeled data: 0.29% on CelebA and 0.39% on LFWA.
• Successfully compensated for data scarcity by incorporating prior knowledge directly into the network.

Conclusions:

• The LSA method offers an effective solution for face attribute recognition with limited labeled data.
• Language-driven prior knowledge can significantly enhance deep learning models in computer vision tasks.
• The approach eliminates the need for computationally expensive pretraining steps, making it more practical for real-world applications.