RL-FM: Reinforcement Learning-Driven Flow Matching for Multimodal Remote Sensing Image Classification

Area of Science:

• Remote Sensing
• Machine Learning
• Computer Vision

Background:

• Multimodal remote sensing image classification integrates data from hyperspectral images (HSI) and light detection and ranging (LiDAR) for improved land-cover pattern recognition.
• Existing models often overlook uncertainties from data acquisition and labeling, leading to reduced robustness and generalization.
• This limitation hinders performance on unseen data due to noise and spurious correlations.

Purpose of the Study:

• To develop a robust multimodal remote sensing image classification framework that accounts for aleatoric and epistemic uncertainties.
• To improve model generalization capabilities, especially in scenarios with limited labeled data.
• To reformulate multimodal joint distribution modeling as a flow matching optimization problem.

Main Methods:

• A Reinforcement Learning-driven Flow Matching (RL-FM) framework is proposed, utilizing variational autoencoders for feature distribution modeling.
• A Gaussian mixture strategy creates an initial distribution, guided by label information for flow matching optimization.
• The distribution evolution is modeled as a Markov decision process (MDP) for optimal path learning, incorporating counterfactual proximal policy optimization (CPPO).

Main Results:

• The RL-FM framework demonstrates state-of-the-art performance on multiple benchmark datasets for multimodal remote sensing image classification.
• The approach effectively mitigates issues related to noise and spurious correlations by addressing data uncertainties.
• Counterfactual causal inference in CPPO enhances generalization with limited labeled samples.

Conclusions:

• The proposed RL-FM framework offers a significant advancement in multimodal remote sensing image classification by effectively handling uncertainties.
• The integration of flow matching, reinforcement learning, and causal inference leads to improved robustness and generalization.
• The method provides a promising direction for future research in high-performance remote sensing data analysis.