Category Name Expansion and an Enhanced Multimodal Fusion Framework for Few-Shot Learning.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Few-shot learning (FSL) is crucial for addressing data scarcity in image processing.
• Existing unimodal FSL methods struggle with fine-grained category distinctions due to limited data.
• Multimodal approaches are needed to enrich semantic representations and improve FSL performance.

Purpose of the Study:

• To propose a novel multimodal few-shot learning method integrating text and image data.
• To enhance the model's ability to capture fine-grained differences between categories.
• To improve the generalization capability of few-shot learning models.

Main Methods:

• Category name expansion and image feature enhancement were employed.
• Integration of expanded text with image features to enrich semantic representation.
• A cross-modal residual connection strategy was introduced for progressive feature fusion and alignment.

Main Results:

• The proposed method demonstrated superior performance on natural image datasets (CIFAR-FS, FC100).
• Effective results were also achieved on a medical image dataset.
• The approach successfully alleviated information bottlenecks and enhanced model generalization.

Conclusions:

• Multimodal fusion, particularly with category name expansion and feature enhancement, significantly boosts few-shot learning.
• The cross-modal residual connection strategy effectively aligns features and maximizes mutual information.
• This method offers a promising solution for data-scarce scenarios in both natural and medical image analysis.