Hyper-MDR: An open-world multimodal reasoning framework based on dynamic hypergraph and meta-strategy optimization.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Open-world object detection (OWOD) is vital for intelligent systems, requiring recognition of known and discovery of unknown categories.
• Existing OWOD methods struggle with shallow cross-modal interactions and rigid reasoning, limiting adaptability to dynamic environments.

Purpose of the Study:

• To develop an advanced OWOD method that addresses limitations in cross-modal interaction and reasoning for dynamic environments.
• To enable deep multimodal fusion and effective knowledge propagation between known and unknown object categories.

Main Methods:

• Hypergraph enhancement: Image regions, text, and semantic prototypes form nodes; a gating network creates hyperedges for high-order dependency modeling.
• Hierarchical hypergraph convolutional network: Facilitates knowledge propagation between known and unknown categories.
• Meta-policy gradient-based adaptive controller: Dynamically adjusts feature fusion, propagation depth, and attention topology.

Main Results:

• The proposed method achieves 76.8% accuracy on the OWOD dataset.
• Demonstrates improved performance in handling continuous emergence of new categories and dynamic modality reliability.

Conclusions:

• The developed method offers a new paradigm for open-world multimodal perception.
• Integrates semantic depth and adaptability for robust object detection in complex, open-world scenarios.