Introducing mCODEGPT as a zero-shot information extraction from clinical free text data tool for cancer research

Area of Science:

• Oncology
• Natural Language Processing
• Data Science

Background:

• Clinical notes contain vast amounts of unstructured cancer patient data.
• Traditional NLP methods for information extraction require extensive expert annotation and model training.
• There is a need for efficient and accurate methods to extract and structure cancer data.

Purpose of the Study:

• To introduce a Large Language Model (LLM)-based tool for zero-shot information extraction from cancer clinical notes.
• To structure extracted data according to the minimal Common Oncology Data Elements (mCODE™) framework.
• To evaluate the effectiveness of hierarchical prompt engineering for improving LLM accuracy.

Main Methods:

• Utilized LLM zero-shot learning capabilities, eliminating the need for expert-annotated data.
• Employed advanced hierarchical prompt engineering strategies to mitigate LLM limitations like token hallucination.
• Tested the approach on 1,000 synthetic clinical notes across various cancer types, comparing it to single-step prompting.

Main Results:

• The hierarchical prompt engineering strategy achieved 94% accuracy with a 5% misidentification/misplacement rate.
• This significantly outperformed the traditional single-step prompt strategy (87% accuracy, 10% misidentification/misplacement rate).
• The method successfully unified diverse cancer staging systems (TNM, FIGO) into a standardized framework, improving stage extraction accuracy.

Conclusions:

• LLMs, guided by structured prompting, can accurately extract complex clinical information without expert-labeled data.
• This approach offers a powerful method for leveraging unstructured clinical notes for cancer research.
• The developed tool demonstrates potential for advancing cancer data standardization and analysis.