Enhancing Infrared Optical Flow Network Computation through RGB-IR Cross-Modal Image Generation

Area of Science:

• Computer Vision
• Machine Learning
• Infrared Imaging

Background:

• Current deep learning for optical flow is limited to RGB images due to challenges in capturing real infrared (IR) optical flow.
• This restricts the application and research of optical flow computation in IR domains.

Purpose of the Study:

• To propose a method for generating an optical flow dataset specifically for IR images.
• To enable deep learning-based optical flow computation on infrared imagery.

Main Methods:

• Utilized an RGB-IR cross-modal image transformation network, based on an improved Pix2Pix implementation.
• Validated the transformation network using the M³FD RGB-IR aligned bimodal dataset.
• Applied the transformation to the KITTI RGB optical flow dataset to generate IR images for training.

Main Results:

• Trained an optical flow computation network using the transformed IR images.
• Analyzed the performance of the optical flow network before and after training on IR data.
• Demonstrated the feasibility of generating and utilizing IR optical flow datasets.

Conclusions:

• The proposed method successfully generates IR optical flow datasets by transforming existing RGB datasets.
• This approach expands the scope of deep learning applications for optical flow computation into the infrared spectrum.
• The study provides a foundation for future research in IR-based computer vision tasks.