Examining the Impact of Optical Aberrations to Image Classification and Object Detection Models

Area of Science:

• Computer Vision
• Deep Learning
• Image Processing

Background:

• Deep neural networks (DNNs) are vital in computer vision, even in safety-critical applications, necessitating robust performance against image disturbances like blur.
• Existing benchmarks often simplify blur, failing to capture the complex kernel shapes from real optical systems, thus limiting accurate robustness evaluation.

Purpose of the Study:

• To develop and introduce realistic blur datasets, OpticsBench and LensCorruptions, for evaluating the robustness of vision models.
• To assess the performance variation of pre-trained models on these new datasets, revealing limitations in current robustness assessments.

Main Methods:

• OpticsBench dataset: Focuses on primary aberrations (coma, defocus, astigmatism) using Zernike polynomials.
• LensCorruptions dataset: Incorporates linear combinations of Zernike polynomials, simulating 100 real-world lenses with diverse optical characteristics.
• Evaluations were conducted on image classification and object detection tasks using ImageNet and MSCOCO datasets.

Main Results:

• Significant performance drops were observed across various pre-trained models on both OpticsBench and LensCorruptions datasets.
• The findings underscore the inadequacy of current blur simulation methods for comprehensive model robustness evaluation.
• The OpticsAugment framework demonstrated improved robustness, achieving substantial performance gains on OpticsBench and common corruptions when used for data augmentation.

Conclusions:

• Realistic optical blur significantly impacts deep neural network performance, necessitating the use of datasets like OpticsBench and LensCorruptions for accurate robustness assessment.
• Employing optical blur kernels in data augmentation strategies, such as with OpticsAugment, can enhance model robustness against real-world image degradations.