Cancer type, stage and prognosis assessment from pathology reports using LLMs

Area of Science:

• Artificial Intelligence in Medicine
• Computational Pathology
• Natural Language Processing

Background:

• Large Language Models (LLMs) show promise in NLP tasks.
• Their use in pathology report analysis is underexplored.
• Key areas include cancer type, staging, and prognosis.

Purpose of the Study:

• Evaluate LLM performance on pathology report analysis.
• Assess capabilities in cancer identification, AJCC staging, and prognosis.
• Develop and compare specialized instruction-tuned models.

Main Methods:

• Evaluated GPT, Mistral, and Llama models in a zero-shot setting.
• Assessed performance on information extraction and reasoning tasks.
• Developed and fine-tuned Path-llama3.1-8B and Path-GPT-4o-mini-FT models.

Main Results:

• Zero-shot performance varied across evaluated LLMs.
• Instruction-tuned models Path-llama3.1-8B and Path-GPT-4o-mini-FT showed superior results.
• These models demonstrated enhanced accuracy in cancer identification, staging, and prognosis.

Conclusions:

• LLMs can be effectively applied to pathology report analysis.
• Specialized, instruction-tuned models offer improved performance.
• Future work could further refine AI tools for digital pathology.