Large Language Model (LLM)-Predicted and LLM-Assisted Calculation of the Spinal Instability Neoplastic Score (SINS)...

Background: The Spinal Instability Neoplastic Score (SINS) guides treatment for patients with spinal tumors, but issues arise with complexity, interobserver variability, and time demands. Large language models (LLMs) may help overcome these limitations. Objectives: This study evaluates the accuracy and efficiency of a privacy-preserving LLM (PP-LLM) for SINS calculation, with and without clinician involvement, to assess its feasibility as a clinical decision-support tool. Methods: This retrospective observational study was granted a Domain-Specific Review Board waiver owing to minimal risk. Patients from 2020 to 2022 were included. A PP-LLM was employed to maintain secure handling of patient data. A consensus SINS reference standard was established by musculoskeletal radiologists and an orthopedic surgeon. Eight orthopedic and oncology trainees were divided into two groups to calculate SINS, with and without PP-LLM assistance. LLM-predicted scores were also generated independently of any human input. Results: The main outcomes were agreement with the reference standard (measured by intraclass correlation coefficients [ICCs]) and time required for SINS calculation. The LLM-assisted method achieved excellent agreement (ICC = 0.993, 95%CI = 0.991-0.994), closely followed by the LLM-predicted approach (ICC = 0.990, 95%CI = 0.984-0.993). Clinicians working without LLM support showed a significantly lower ICC compared to both LLM methods (0.968, 95%CI = 0.960-0.975) (both p < 0.001). The LLM alone produced scores in approximately 5 s, while the median scoring time for LLM-assisted clinicians was 60.0 s (IQR = 46.0-80.0), notably shorter than the 83.0 s (IQR = 58.0-124.0) required without LLM assistance. Conclusions: An LLM-based approach, whether used autonomously or in conjunction with clinical expertise, enhances both accuracy and efficiency in SINS calculation. Adopting this technology may streamline oncologic workflows and facilitate more timely interventions for patients with spinal metastases.