Generative Model-Based Fusion for Improved Few-Shot Semantic Segmentation of Infrared Images

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Infrared (IR) imaging is crucial for autonomous driving, fire safety, and defense.
• Semantic segmentation of IR images is challenging due to data scarcity, low contrast, and novel class emergence.
• Existing few-shot segmentation (FSS) models for IR images often require paired visible RGB data, which is impractical in many applications.

Purpose of the Study:

• To develop new strategies for few-shot segmentation (FSS) of infrared (IR) images without relying on paired visible RGB data.
• To address challenges in IR image semantic segmentation, including data scarcity and limited contrast.
• To improve the performance of FSS models in real-world IR imaging scenarios.

Main Methods:

• Utilized generative modeling for synthesizing auxiliary data to enhance channel information and IR data for augmentation.
• Developed a novel fusion ensemble module to integrate different modalities and improve the relationship between support and query sets.
• Employed generative techniques to overcome data scarcity and limited contrast in IR images for FSS.

Main Results:

• The proposed methods successfully perform few-shot segmentation on IR images without paired RGB data.
• Synthesized auxiliary data improved the FSS model's ability to capture relationships between support and query sets.
• IR data synthesis effectively addressed data scarcity, leading to improved segmentation accuracy.
• The novel fusion ensemble module further enhanced performance by integrating multi-modal information.

Conclusions:

• The developed generative modeling and fusion techniques offer a robust solution for few-shot segmentation of IR images.
• These strategies effectively overcome limitations of existing FSS models, particularly the need for paired RGB data.
• The approach shows significant improvements over state-of-the-art methods on various IR datasets, enabling broader applications of IR image analysis.