Deep learning generalization study on optical coherence tomography image denoising

Objective.Noise is a key factor determining imaging quality for optical coherence tomography (OCT). Although deep learning has emerged as an effective denoising method, its generalization capability remains limited, especially when test noise levels deviate from the training data.Approach.To solve this problem, we propose a mixed-training strategy combined with a multi-noise level dataset, aiming to enhance model adaptability to unseen noise conditions. The datasets were constructed by inserting different optical attenuators (4 dB, 6 dB, and 10 dB) in the sample arm of an SS-OCT system, simulating diverse noise scenarios. Our mixed-training strategy unified images from 0 dB, 6 dB, and 10 dB into a composite training set, while a portion of the 4 dB data was used as an independent test set to evaluate generalization capability. This approach was applied to classical denoising networks including residual network, U-shape network, denoising convolutional neural network, and attention-guided deformable convolutional network, under both supervised and unsupervised frameworks.Main results.Experimental results demonstrated that models trained with the mixed-training strategy achieved robust performance across noise levels, including unseen 4 dB noise images, where the Unet model trained with the mixed-training strategy under the supervised learning framework attained a peak signal-to-noise ratio (PSNR) of 29.233 dB and an structural similarity index measure (SSIM) of 0.807. This performance is close to that of dedicated models trained on 4 dB data, which achieved a PSNR of 29.221 dB and an SSIM of 0.809. Visual and numerical evaluations further confirmed that the mixed-trained networks effectively suppressed noise artifacts, even under mismatched noise conditions.Significance.This work confirms the value of multi-noise level datasets and introduces a mixed-training strategy that enhances generalization and supports reliable OCT analysis in practice.