Cross Task Modality Alignment Network for Sketch Face Recognition.

Area of Science:

• Computer Science
• Artificial Intelligence
• Biometrics

Background:

• Sketch face recognition involves matching facial images across different modalities (sketch to photo), crucial for criminal investigations.
• Existing methods focus on feature alignment but neglect the significant challenge posed by small sample sizes, limiting performance.

Purpose of the Study:

• To propose an effective Cross Task Modality Alignment Network (CTMAN) to enhance sketch face recognition performance.
• To address the small sample problem inherent in sketch face recognition tasks.

Main Methods:

• Introduced a meta-learning training episode strategy to simulate few-shot learning scenarios.
• Developed a two-stream network for modality alignment embedding learning, capturing both modality-specific and shared features.
• Incorporated two cross-task memory mechanisms for collecting negative features and a cross-task modality alignment loss for improved training.

Main Results:

• The proposed CTMAN significantly outperforms existing state-of-the-art methods.
• Demonstrated superior performance across multiple benchmark datasets: UoM-SGFSv2 (set A and B), CUFSF, and PRIP-VSGC.

Conclusions:

• The CTMAN effectively bridges the cross-modality gap in sketch face recognition, particularly under small sample conditions.
• The proposed meta-learning strategy and network architecture offer a robust solution for challenging biometric identification tasks.