Deep learning for oncologic treatment outcomes and endpoints evaluation from CT scans in liver cancer

Accurate treatment response assessment using serial CT scans is essential in oncological clinical trials. However, oncologists' assessment following the Response Evaluation Criteria in Solid Tumors (RECIST) guideline is subjective, time-consuming, and sometimes fallible. Advanced liver cancer often presents multifocal hepatic lesions on CT imaging, making accurate characterization more challenging than with other malignancies. In this work, we developed a tumor volume guided comprehensive objective response evaluation based on deep learning (RECORD) for liver cancer. RECORD performs liver tumor segmentation, followed by sum of the volume (SOV)-based treatment response classification and new lesion assessment. Then, it can provide treatment evaluations of response, stability, and progression, and calculates progression-free survival (PFS) and response time. The RECORD pipeline was developed with both CNN and ViT backbones. Its performance was evaluated in three longitudinal cohorts involving 60 multi-national centers, 206 patients, 891 CT scans, using internal five-fold cross-validation and external validations. RECORD with the most effective backbone achieved an average AUC-response of 0.981, AUC-stable of 0.929, and AUC-progression of 0.969 for SOV-based disease status classification, F1-score of 0.887 for new lesion identification, and accuracy of 0.889 for final treatment outcome assessments across all cohorts. RECORD's PFS and response time predictions strongly correlated with clinician's assessments (P < 0.001). Moreover, RECORD can better stratify high-risk versus low-risk patients for overall survival compared to the human-assessed RECIST results. In conclusion, RECORD demonstrates efficiency and objectivity in analyzing liver lesions for treatment response evaluation. Further research should extend the pipeline to other metastatic organ sites.