[Topology-aware self-attention network for automatic segmentation of oral potentially malignant disorder images]

Objective: To develop an automatic oral lesion segmentation model (TopoFormer) that integrates topological data analysis (TDA) with a Transformer architecture, thereby improving the accuracy and robustness of lesion segmentation for oral potentially malignant disorders (OPMD) and oral squamous cell carcinoma (OSCC). Methods: Experiments were conducted on a public comprehensive oral cavity image dataset released by Piyarathne et al., containing 2 271 oral white-light photographs from 623 patients and covering three categories: At the patient level, the data were stratified and randomly split into training, validation, and test sets with a ratio of 0.8/0.1/0.1. SegFormer was used as the backbone network, and a self-designed topological attention module (TAM) was embedded in the deep feature space. TAM leveraged persistent homology to extract robust topological structures from feature maps and dynamically suppressed low-persistence noise. Performance was evaluated using the Dice similarity coefficient (DSC), mean intersection over union (mIoU), mean pixel accuracy (mPA), 95% Hausdorff distance (HD95), and Betti number error (BNE), and was compared with mainstream segmentation networks (U-Net, DeepLabV3+, and SegFormer). Results: On the test set, TopoFormer achieved a DSC of 0.801, an mIoU of 71.1%, an mPA of 82.31%, reduced HD95 to 9.62 pixels, and obtained a BNE of 1.28. Compared with the baseline model, the proposed model effectively reduced false positives caused by specular reflections and produced smoother and more continuous lesion boundaries. Conclusions: TopoFormer demonstrates higher accuracy and robustness in segmenting OPMD and OSCC lesions. Topological priors can enhance segmentation performance in complex oral environments, indicating use potential in computer-aided diagnosis.