A denoising model based on multi-agent reinforcement learning with data transformation for digital tomosynthesis

Objective. Denoising models based on the supervised learning have been proposed for medical imaging. However, its clinical availability in digital tomosynthesis (DT) imaging is limited due to the necessity of a large amount of training data for providing acceptable image quality and the difficulty in minimizing a loss. Reinforcement learning (RL) can provide the optimal pollicy, which maximizes a reward, with a small amount of training data for implementing a task. In this study, we presented a denoising model based on the multi-agent RL for DT imaging in order to improve the performance of the machine learning-based denoising model.Approach. The proposed multi-agent RL network consisted of shared sub-network, value sub-network with a reward map convolution (RMC) technique and policy sub-network with a convolutional gated recurrent unit (convGRU). Each sub-network was designed for implementing feature extraction, reward calculation and action execution, respectively. The agents of the proposed network were assigned to each image pixel. The wavelet and Anscombe transformations were applied to DT images for delivering precise noise features during network training. The network training was implemented with the DT images obtained from the three-dimensional digital chest phantoms, which were constructed by using clinical CT images. The performance of the proposed denoising model was evaluated in terms of signal-to-noise ratio (SNR), structural similarity (SSIM) and peak signal-to-noise ratio (PSNR).Main results. Comparing the supervised learning, the proposed denoising model improved the SNRs of the output DT images by 20.64% while maintaining the similar SSIMs and PSNRs. In addition, the SNRs of the output DT images with the wavelet and Anscombe transformations were 25.88 and 42.95% higher than that for the supervised learning, respectively.Significance. The denoising model based on the multi-agent RL can provide high-quality DT images, and the proposed method enables the performance improvement of machine learning-based denoising models.