Radiologist-guided multimodal fusion network for differentiating parotid tumors on 3D non-contrast CT

To improve the diagnostic accuracy of benign versus malignant parotid tumors using non-contrast CT. We propose a clinician-in-the-loop hybrid framework that mitigates low soft-tissue contrast by fusing semi-automated 3D visual features with expert semantic descriptors.

This multi-center retrospective study analyzed non-contrast CTs from 367 patients. We developed AttenNet, a semi-automated model. First, an nnU-Net module automates 3D tumor segmentation, eliminating manual bounding boxes. A 3D-ResNet backbone then extracts volumetric features, which are dynamically fused with radiologist-provided semantic labels via a Multi-Head Cross-Attention Fusion (MHAF) mechanism. This allows the network to query visual morphology based on expert priors. Diagnostic performance and Gradient-weighted Class Activation Mapping (Grad-CAM) interpretability were evaluated.

The segmentation achieved a median Dice of 0.827. AttenNet (3D-ResNet) yielded an AUC of 0.908, significantly outperforming the image-only 3D-ResNet (AUC: 0.741) and expert-feature models (AUC: 0.821). External validation (n = 20) reached an AUC of 0.560-0.910.However, given the extremely small sample size, these results must be interpreted strictly as an exploratory proof-of-concept rather than definitive confirmation of generalizability. Grad-CAM confirmed that AttenNet highlights clinically essential geometric indicators-specifically tumor margins and internal heterogeneity-for differentiating malignancy, whereas single-modality baselines focused on background artifacts.

This clinician-in-the-loop paradigm successfully integrates spatial imaging with expert reasoning. While multimodal fusion drove the primary performance gains, the MHAF mechanism provided enhanced spatial interpretability.