Hypergraph Mamba Reasoning-Based Social Relation Recognition.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Recognizing social relations from images is vital for advancing machine perception of social interactions.
• Existing methods primarily focus on single-type relations, struggling with complex, hybrid social connections in real-world scenarios.
• Hybrid social relation recognition involves multipartite interactions beyond simple dyadic relationships.

Purpose of the Study:

• To propose a novel Hypergraph Mamba (HGM) framework for effectively modeling and recognizing complex, hybrid social relations from images.
• To address the limitations of current methods in handling high-order, multipartite interactions inherent in real-world social scenarios.
• To enhance the accuracy and robustness of social relation recognition by capturing intricate interdependencies.

Main Methods:

• Developed a Hypergraph Mamba (HGM) framework utilizing Person-Person Hypergraphs (PPH) and Person-Object Hypergraphs (POH) to model multipartite interactions.
• Incorporated a Vertex Selection Algorithm to filter confounders and mitigate inference confusion.
• Implemented a Vertex Interaction Operator for optimal global vertex neighborhood discovery and a Multilevel Transformer for adaptive knowledge-visual signal fusion.

Main Results:

• The HGM model demonstrated superior accuracy in predicting social relations across multiple public datasets.
• Extensive ablation studies confirmed the effectiveness of the proposed HGM framework and its components.
• The model significantly outperformed state-of-the-art methods in complex social relation recognition tasks.

Conclusions:

• The proposed HGM framework effectively models high-order multipartite interactions for social relation recognition.
• HGM offers a robust and accurate solution for recognizing complex hybrid social relations, outperforming existing approaches.
• The framework provides a significant advancement in machine perception of social interactions, with potential applications in various AI domains.