Related Experiment Video
Updated: Aug 11, 2026
Multimodal Cross-Device and Marker-Free Co-Registration of Preclinical Imaging Modalities
Published on: October 27, 2023
BMR-Restormer: A transformer-based blind inpainting network for ultrasound marker removal
Jiabin Yu1, Hongmei Mi2, Dong Chen2
1Shanghai Skin Disease Hospital, School of Medicine & Frontier Science Center for Stem Cell Research, Bioinformatics Department, School of Life Sciences and Technology, Tongji University, Shanghai, 200092, China; College of Information Engineering, China Jiliang University, Hangzhou, Zhejiang, 310018, China; Taizhou Campus, Zhejiang Cancer Hospital (Taizhou Cancer Hospital), Taizhou, Zhejiang, 317502, China; Taizhou Key Laboratory of Minimally Invasive Interventional Therapy & Artificial Intelligence, Taizhou Campus of Zhejiang Cancer Hospital (Taizhou Cancer Hospital), Taizhou, Zhejiang, 317502, China.
Background And Objectives:
Artificial markers, including crosses, text, and measurement lines, are commonly embedded in clinical ultrasound images to aid visual diagnosis. However, these overlays introduce substantial bias in deep learning models by drawing attention away from pathological structures, leading to degraded performance in computer-aided diagnosis (CAD).
Methods:
To address this issue, we propose the BMR-Restormer network specifically optimized for ultrasound marker removal. The architecture integrates Multi-Group Transposed Attention (MGTA) and Masked Window Self-Attention (MWSA) to enhance local-global feature modeling under low-contrast and speckle-rich conditions. A progressive learning strategy was employed to support coarse-to-fine feature acquisition.
Results:
Our method achieved state-of-the-art image restoration performance, with PSNR/SSIM/LPIPS values of 63.68/0.999/0.003 on the internal dataset and 59.30/0.999/0.005 on the external dataset. To evaluate the clinical impact of artificial markers, we conducted three downstream tasks (classification, segmentation, and detection) under different training-testing configurations. The results showed that models trained on marker-containing data had markedly lower generalization performance when applied to marker-free images. For example, classification precision dropped from 0.8830 to 0.5484 when the model trained on marker-containing data was evaluated on clean images. Similarly, segmentation Dice scores fell from 0.8217 to 0.4759. In object detection, AP@50 decreased from 0.9928 to 0.2393.
Conclusions:
These results demonstrated that artificial markers cause models to overfit spurious features and shift attention away from diagnostically relevant regions. In contrast, models trained on marker-free data maintained high accuracy on both clean and marked images, confirming that the proposed BMR-Restormer provides a robust and clinically reliable solution for ultrasound marker removal.
