Image-to-image machine translation enables computational defogging in real-world images

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Machine learning for computational defogging shows promise but is limited by a lack of large-scale, real-world foggy image datasets.
• Existing datasets often lack sufficient dense fog conditions or rely on synthetic data, hindering model generalization.

Purpose of the Study:

• To introduce Stereofog, an open-source dataset of real-world paired clear and foggy images, with a focus on dense fog conditions.
• To evaluate the performance of machine learning models, specifically image-to-image translation, for computational defogging using the Stereofog dataset.

Main Methods:

• Developed a binocular imaging system to capture paired clear and foggy images.
• Created Stereofog, an open-source dataset with 10,067 image pairs, emphasizing dense fog (Laplacian variance, vL < 10).
• Trained a pix2pix image-to-image translation model on Stereofog and compared its performance against models trained on synthetic data.

Main Results:

• The pix2pix model trained on Stereofog achieved a complex wavelet structural similarity index (CW-SSIM) > 0.7 and peak signal-to-noise ratio (PSNR) > 17 under dense fog.
• Models trained on Stereofog significantly outperformed those trained on synthetic or augmented data, particularly in dense fog scenarios.
• Performance analysis identified limitations including dataset diversity and hallucinations, common in machine learning defogging.

Conclusions:

• Stereofog is a valuable resource for advancing machine learning-based computational defogging research.
• Real-world datasets like Stereofog are crucial for developing robust defogging algorithms that perform well under diverse and dense fog conditions.
• The study highlights the potential of machine learning for defogging while underscoring the need for further research into dataset diversity and model generalization.