Semantic Interaction Meta-Learning Based on Patch Matching Metric.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Metric-based meta-learning excels in few-shot image classification but is limited by metric choice and feature representation.
• Holistic image features in current methods can cause "supervision collapse", ignoring critical task-specific details.
• Relying solely on visual features is insufficient for characterizing support classes, especially with limited samples.

Purpose of the Study:

• Introduce Patch Matching Metric-based Semantic Interaction Meta-Learning (PatSiML) to overcome limitations in few-shot image classification.
• Develop a patch matching metric strategy to counteract supervision collapse and improve feature representation.
• Integrate semantic knowledge with visual features to enhance classification accuracy.

Main Methods:

• Utilize a Transformer-based patch matching metric strategy to generate distinct patch embeddings from input images.
• Employ a graph convolutional network to dynamically create task-specific embeddings for precise matching between support and query image patches.
• Integrate a label-assisted channel semantic interaction strategy, merging word embeddings with patch-level visual features via a language model.

Main Results:

• PatSiML demonstrates significant accuracy improvements across four diverse datasets.
• Achieved classification accuracy gains ranging from 0.65% to 21.15% compared to existing methodologies.
• The framework effectively combines semantic understanding with visual information for robust classification.

Conclusions:

• PatSiML offers a robust and effective solution for few-shot image classification.
• The proposed methods successfully address supervision collapse and insufficiency of visual features.
• The integration of semantic information significantly boosts classification performance in low-data regimes.