A lightweight hybrid CNN-transformer model for smoke removal in laparoscopic surgery images

In laparoscopic surgery, which is a minimally invasive procedure, endoscopic cameras are very common in visual guidance. Nevertheless, surgical smoke produced during electrocautery and laser ablation greatly impairs the quality of images produced by decreasing contrast, blurring edges, and distorting color data. This deterioration is experienced with both the visibility of the surgeon and computer vision-based surgical assistance system performance. Physics-based and image enhancement models are ineffective due to unrealistic assumptions of the smoke distribution, whereas deep learning methods are typically complicated.

It is based on the Progressive Frequency-Aware Network (PFAN), which is a lightweight CNN-Transformer hybrid that isolates smoke and tissue signal in the frequency domain. PFAN uses Multi-scale Bottleneck-Inverting (MBI) blocks to restore high-frequency details and Locally-Enhanced Axial Attention Transformer (LAT) blocks to learn global contextual details. In order to improve the quality of reconstruction and structural preservation, we suggest another Hybrid Attention-Residual UNet model that incorporates residual learning and dual attention (channel and spatial attention) to a conditional GAN. To enhance the elimination and localization of smoke compared to PFAN, the suggested hybrid model aims at spatial refinement of features and adaptive weighting of features.

The experimental findings prove that the suggested hybrid model can be competitive in terms of PSNR and SSIM, but with the preservation of structural consistency and enhanced visual clarity.

Frequency-aware learning with attention-based refinement is a solid solution to laparoscopic image de-smoking.