Multimodal Language Model for Jaw Osteonecrosis Diagnosis and Treatment

Owing to the complexity of the clinical manifestations in patients with osteonecrosis of the jaw (ONJ), particularly in settings lacking oral and maxillofacial surgery, timely and accurate treatment remains challenging. Here, we developed an integrated vision-language interactive system (namely, the Osteonecrosis of the Jaw Treatment Supporting [ONJ-TS] system), which is designed to provide clinical support for the diagnosis and treatment of ONJ. We retrospectively collected panoramic radiography (PR) images and medical records of 366 patients with ONJ to develop the ONJ-TS system. In the ONJ-TS system, the core Qwen2-VL was fine-tuned for natural language question answering on ONJ treatment. To support the primary functionality of the Qwen2-VL, we trained the SwinTransformer and the U-Net to identify ONJ cases and segment the ONJ regions in the PR images, respectively. Human evaluation of the output responses indicated that the ONJ-TS system exceeded the accuracy levels of junior oral and maxillofacial surgeons (OMFSs) with accuracy scores of 0.9598 (95% confidence interval [CI], 0.9351-0.9845) and 0.8835 (95% CI, 0.8432-0.9238), respectively (P = 0.003). Notably, in comparative analyses with the senior OMFSs (accuracy, 0.9959 [95% CI, 0.8432-0.9238]), the ONJ-TS system demonstrated comparable efficacy in disease management (P < 0.001). Extensive experiments showed that the ONJ-TS system outperformed baseline models in generating accurate responses to treatment support. Given its multifaceted capabilities, the ONJ-TS system holds promise as a digital solution for supporting clinical decisions in managing ONJ and enhancing clinical efficiency and treatment accuracy.