Integrating a Shareable Artificial Intelligence Model Into Clinical Research for Cancer Recurrence in Patients With...

Cancer recurrence in clinical settings is documented in unstructured text, requiring labor-intensive manual record review to extract this outcome. A shareable natural language processing model developed at Dana-Farber Cancer Institute (DFCI)-DFCI-imaging-student-efficiently extracts cancer outcomes from radiology reports. We applied this model in a community oncology setting, aggregating report-level predictions to derive patient-level outcomes, and evaluated its performance in determining recurrence and time-to-recurrence in patients with breast cancer (BC) or colorectal cancer (CRC).

We randomly sampled 200 patients with BC and 200 patients with CRC from two cohorts at Kaiser Permanente Northern California. Patients were diagnosed with stage III disease (2005-2019) and followed until July 31, 2024, death, or disenrollment. We manually reviewed recurrence (local/regional/distant), recurrence date, and sites of recurrence using oncology, radiology, and pathology information in electronic health records. We then applied the DFCI-imaging-student model to radiology reports and compared recurrence based on the model outcomes against manual review.

A total of 7,195 radiology reports were processed. During a median follow-up of 8.4 years for BC and 6.8 years for CRC, manual review identified 78 recurrence cases in BC (39%) and 70 in CRC (35%). The DFCI-imaging-student model demonstrated high sensitivity and specificity for recurrence detection in both cancers (breast: 92.3% and 92.6%, CRC: 94.3% and 86.9%) and moderate-to-high accuracy in identifying the sites of distant metastasis. Among true positives, the median error in time-to-recurrence was 0.16 months for breast and 0.48 months for CRC.

Outcomes derived from the DFCI-imaging-student model output demonstrated high accuracy, providing an efficient determination of recurrence and time-to-recurrence in large-scale research to improve recurrence surveillance and facilitate collaborative research.