Multimodal Fusion of Intraoperative FLIm and Preoperative PET/CT for Patient-Level Prediction of Lymph Node...

Metastatic lymph node (MLN) detection remains a major clinical challenge in head and neck cancer, as nodal involvement is strongly associated with poor prognosis and directly affects treatment planning. Previous approaches typically rely on cropped lymph node (LN) regions or tumor contours for MLN identification, requiring substantial expert annotation during preprocessing and relying solely on imaging information. As a result, small or low-contrast metastatic nodes may be missed, while benign lymph nodes may be incorrectly identified as metastatic due to overlapping imaging characteristics. To address these limitations, we propose a multimodal learning framework that integrates anatomical and metabolic features from head and neck PET/CT images with biochemical features derived from FLIm for patient-level MLN prediction, without requiring manual lymph node cropping or tumor contouring during inference.

To enable robust imaging representation learning, a region-aware PET/CT network based on a merging-diverging architecture was first pretrained on the HECKTOR 2022 dataset and then fine-tuned on the institutional cohort. In parallel, FLIm point-wise measurements with clinical variables were encoded using a multilayer perceptron (MLP) and aggregated into subject-level representations. To effectively combine these modalities, two multimodal fusion strategies were evaluated at the decoder stage, including cube-based fusion and squeeze-and-excitation (SE)-based fusion. The proposed strategies were evaluated on a cohort of 53 patients.

Compared with the single-modality baselines, both multimodal fusion strategies achieved better patient-level MLN prediction. The PET/CT-only segmentation-driven model and FLIm-only model reached balanced accuracies of 0.815 and 0.665, with AUCs of 0.828 and 0.614, respectively. Cube-based fusion improved balanced accuracy and AUC to 0.827 and 0.850, respectively, while channel-wise SE-based fusion achieved the best overall performance, with a balanced accuracy of 0.839 and an AUC of 0.872.

These results suggest that multimodal integration may improve patient-level MLN prediction compared with single-modality approaches. Given the limited sample size, these findings should be interpreted as hypothesis-generating and require validation in larger, independent patient cohorts.