Enabling Just-in-Time Clinical Oncology Analysis With Large Language Models: Feasibility and Validation Study Using...

Area of Science:

• Oncology research
• Artificial intelligence in medicine
• Clinical data analysis

Background:

• Traditional cancer registries are labor-intensive and hinder timely research.
• Large language models (LLMs) show promise for automating data extraction.
• The potential of LLMs for direct, just-in-time (JIT) analysis of clinical narratives is underexplored.

Purpose of the Study:

• To evaluate Gemini 2.5 Pro's ability to enable a JIT clinical oncology analysis paradigm.
• Assess LLM performance in high-fidelity data extraction, complex clinical query answering, automated survival analysis, and hypothesis generation from free-text clinical data.

Main Methods:

• Utilized a synthetic dataset of 240 unstructured clinical letters for stage IV non-small cell lung cancer (NSCLC).
• Evaluated Gemini 2.5 Pro on four JIT capabilities: multiparameter extraction, direct query answering, survival analysis with code generation, and hypothesis generation.
• Measured performance using accuracy, numerical deviation, log-rank P value, Harrell concordance index, and qualitative hypothesis evaluation.

Main Results:

• LLM achieved >99% accuracy in data extraction, comparable to human extraction but significantly faster.
• LLM answered complex queries with <1.5% deviation and autonomously performed end-to-end survival analysis, generating statistically indistinguishable Kaplan-Meier curves.
• LLM generated biologically plausible and potentially novel hypotheses, with human-in-the-loop needed for AI-flagged ambiguities.

Conclusions:

• Frontier LLMs like Gemini 2.5 Pro can perform high-fidelity data extraction, querying, and survival analysis directly from unstructured text.
• This study provides a proof of concept for JIT clinical analysis, potentially revolutionizing oncology research.
• Validation on real-world clinical data is crucial before clinical implementation.