EMeRALDS: Electronic Medical Record Driven Automated Lung Nodule Detection and Classification in Thoracic CT Images

Lung cancer is a leading cause of cancer-related mortality worldwide, primarily due to delayed diagnosis and poor early detection. This study aims to develop a computer-aided diagnosis (CAD) system that leverages large vision-language models (VLMs) for the accurate detection and classification of pulmonary nodules in computed tomography (CT) scans. We propose an end-to-end CAD pipeline consisting of two modules: (i) a detection module (CADe) based on the Segment Anything Model 2 (SAM2), in which the standard visual prompt is replaced with a text prompt encoded by CLIP (Contrastive Language-Image Pretraining), and (ii) a diagnosis module (CADx) that calculates similarity scores between segmented nodules and radiomic features. Synthetic electronic medical records (EMRs) generated in response to radiomic evaluations done by skilled radiologists were used to add clinical context and used with similarity scores to achieve the final classification. The proposed method was experimented on a publicly accessible LIDC-IDRI dataset (1018 CT scans). The proposed method performed well in zero-shot settings for lung nodule analysis. The CADe module had a Dice score of 0.92 and IoU of 0.85 in nodule segmentation. The module of CADx achieved 0.97 specificity in malignancy classification, which is higher than the currently available fully supervised methods. The integration of VLMs with radiomics and synthetic EMRs allows for accurate and clinically relevant CAD of pulmonary nodules in CT scans. The proposed system shows strong potential to enhance early lung cancer detection, increase diagnostic confidence, and improve patient management in routine clinical workflows.