MECOM: A Meta-Completion Network for Fine-Grained Recognition With Incomplete Multi-Modalities.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Fine-grained recognition with incomplete multi-modal data is a challenging problem.
• Existing methods often overlook the issue of missing modalities.
• There is a need for models that can adapt to and complete missing data.

Purpose of the Study:

• To propose a meta-learning strategy for handling incomplete multi-modal data in fine-grained recognition.
• To develop a method (MECOM) for multimodal fusion and missing modality completion.
• To improve fine-grained recognition accuracy by incorporating part-level features.

Main Methods:

• Leveraging meta-learning for fast modal adaptation and missing modality completion.
• Developing the MECOM method with cross-modal attention and decoupling reconstruction.
• Introducing a partial stream and part-level feature selection for enhanced fine-grained recognition.

Main Results:

• MECOM demonstrates superiority over competing methods in quantitative and qualitative experiments.
• The proposed approach effectively handles incomplete multi-modal data.
• Experiments were conducted on fine-grained and generic multimodal datasets.

Conclusions:

• The proposed meta-learning strategy and MECOM method are effective for fine-grained recognition with incomplete multi-modal data.
• MECOM successfully addresses the challenge of missing modalities.
• The inclusion of part-level features further boosts recognition accuracy.