Data Acquisition and Preparation for Dual-Reference Deep Learning of Image Super-Resolution

Area of Science:

• Computer Vision
• Machine Learning
• Image Processing

Background:

• Deep learning-based image super-resolution (SR) models require accurate training data.
• Synthetically generated low-resolution (LR) and high-resolution (HR) image pairs do not accurately represent real-world camera sampling.
• This discrepancy leads to poor performance of SR models when applied to real images.

Purpose of the Study:

• To develop a novel data acquisition process for creating realistic LR-HR image pairs for training SR models.
• To enhance the accuracy and effectiveness of deep convolutional neural network (DCNN) SR models for real-world applications.

Main Methods:

• A new data acquisition process using real cameras to capture LR-HR image pairs by displaying images on an ultra-high quality screen.
• A spatial-frequency dual-domain registration method for precise sub-pixel alignment of LR-HR pairs.
• Utilizing dual references (captured HR image and original digital image) to strengthen supervised learning.

Main Results:

• Training DCNN SR models with the proposed dataset significantly improved image quality compared to existing datasets.
• The novel data acquisition method demonstrated superior performance for super-resolution tasks.
• The proposed method provides a practical, automated, low-cost solution for tailoring SR models to specific cameras.

Conclusions:

• The developed data acquisition and registration method yields more appropriate training data for image super-resolution.
• This approach effectively addresses the limitations of synthetic data, leading to enhanced SR model performance on real-world images.
• The automated and cost-effective nature of the process makes it a valuable tool for camera-specific SR model development.