IR-NeLF: infrared neural light field on mobile devices

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Infrared imaging presents unique challenges for 3D reconstruction due to indistinct features and poor visibility.
• Existing methods struggle with occlusion and low-detail environments, limiting their applicability.
• Fast rendering on edge devices is crucial for real-world applications.

Purpose of the Study:

• To develop an efficient and robust method for infrared 3D reconstruction.
• To enable fast infrared image generation on mobile and edge devices.
• To improve 3D reconstruction quality under challenging infrared imaging conditions.

Main Methods:

• Proposed Infrared Neural Light Field (IR-NeLF) for infrared 3D reconstruction.
• Developed an infrared image enhancement strategy to recover camera pose using Structure-from-Motion.
• Optimized network architecture for low-frequency infrared image characteristics and incorporated a hybrid loss function.

Main Results:

• Achieved efficient rendering on edge devices, generating a 768x1024 infrared image in 53.6ms on AIpro.
• Demonstrated significantly faster performance compared to MobileR2L (83.3ms) and traditional NeRF (1360s).
• Outperformed baseline methods in PSNR, SSIM, and LPIPS metrics, indicating superior reconstruction quality.

Conclusions:

• IR-NeLF provides an efficient and high-quality solution for infrared 3D reconstruction.
• The method is effective under challenging conditions like occlusion and poor visibility.
• A new high-resolution infrared image dataset was contributed to advance future research.