Graphicalized vision-language modeling for comprehensive lung nodule analysis and risk stratification.

Area of Science:

• Medical Imaging and Artificial Intelligence
• Computational Pathology
• Radiomics and AI

Background:

• Lung cancer diagnosis requires integrating imaging (CT, PET/CT) and text data for tasks like nodule detection and survival prediction.
• Current systems often handle these tasks in isolation, missing interdependencies and potential for synergistic improvement.
• A unified approach is needed to leverage multimodal data for comprehensive lung cancer assessment.

Purpose of the Study:

• To develop VITALIS, a multimodal vision-language framework for integrated lung cancer diagnostics and prognostics.
• To fuse CT/PET/CT imaging with radiology text using advanced AI techniques.
• To enable accurate, individualized, continuous-time risk modeling for lung cancer patients.

Main Methods:

• VITALIS employs a graph-aware Transformer to fuse multimodal data (CT, PET/CT, text).
• Laplacian diffusion and attention mechanisms enrich features and focus on relevant anatomical/clinical contexts.
• A continuous-time latent risk process modeled by Neural ODEs generates patient representations for downstream tasks.

Main Results:

• The framework achieved accurate nodule detection and classification.
• It provided low-false-positive localization and calibrated survival risk estimates.
• Consistent nodule counts were obtained across integrated tasks.

Conclusions:

• Coupling graph-aware multimodal encoding with continuous-time latent dynamics offers a unified approach for lung cancer analysis.
• VITALIS demonstrates the potential for integrated diagnostic and prognostic modeling.
• This framework advances AI-driven precision medicine in oncology.