Multi-Level Transformer-Based Social Relation Recognition

Area of Science:

• Computer Science
• Artificial Intelligence
• Human-Computer Interaction

Background:

• Social relationships are crucial for understanding human behavior and developing intelligent systems.
• Existing social relation recognition (SRR) methods struggle to comprehensively integrate features of varying importance.
• A need exists for advanced SRR frameworks that can effectively fuse multi-scale information.

Purpose of the Study:

• To propose a novel Multi-level Transformer-Based Social Relation Recognition (MT-SRR) framework.
• To enhance the orchestration of multi-scale features for more accurate SRR.
• To address the challenge of data imbalance in social relation recognition tasks.

Main Methods:

• Utilized a Vision Transformer (ViT) for global feature extraction.
• Introduced an Intra-Relation Transformer (Intra-TRM) for dynamic feature fusion.
• Employed an Inter-Relation Transformer (Inter-TRM) to leverage logical constraints among relationships.
• Incorporated a margin-based loss function to mitigate data imbalance.

Main Results:

• The MT-SRR framework demonstrated superior fusion of multi-scale features.
• The model effectively ameliorated the negative impact of data imbalance.
• Experimental results showed significant improvements over state-of-the-art methods on benchmark datasets.

Conclusions:

• MT-SRR offers a robust approach for social relation recognition by effectively integrating multi-level features.
• The proposed framework advances the development of more sophisticated social intelligent systems.
• MT-SRR provides a significant performance improvement in recognizing social relationships, even with imbalanced data.