Multimodal Fusion Image Stabilization Algorithm for Bio-Inspired Flapping-Wing Aircraft.

Area of Science:

• Robotics
• Computer Vision
• Aerospace Engineering

Background:

• Aerial vehicles, particularly flapping-wing platforms, suffer from severe vibrations impacting image quality.
• Existing video stabilization methods often struggle with intense dynamic jitter and rolling shutter distortions.

Purpose of the Study:

• To introduce FWStab, a specialized dataset for flapping-wing video stabilization.
• To propose a multimodal signal fusion framework for enhancing video stability in aerial vehicles.
• To address poor image acquisition quality caused by severe vibrations.

Main Methods:

• Developed a multimodal signal fusion framework integrating image and Inertial Measurement Unit (IMU) sensor data.
• Employed Long Short-Term Memory (LSTM) networks for temporal modeling and smooth trajectory prediction.
• Utilized unsupervised learning with a joint loss function combining pose smoothness and optical flow residuals.
• Implemented a multi-stage training strategy for broad scenario adaptability.

Main Results:

• The proposed framework effectively predicts smooth camera poses by fusing multimodal data.
• Achieved optimized inter-frame stability by warping sensor-estimated motion to the network-predicted trajectory.
• Maintained global scene motion rigidity and rectified rolling shutter distortions.
• Demonstrated remarkable stabilization adaptability across diverse flight scenarios.

Conclusions:

• The multimodal fusion framework significantly improves video stabilization for flapping-wing platforms.
• The FWStab dataset provides a valuable resource for developing and evaluating advanced stabilization techniques.
• The unsupervised learning approach offers an effective solution for complex aerial videography challenges.