Verification is All You Need: Prompting Large Language Models for Zero-Shot Clinical Coding.

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Natural Language Processing

Background:

• Clinical coding translates Electronic Health Record (EHR) data into structured codes like ICD-10 for healthcare applications.
• Current deep learning models struggle with generalizability due to data scarcity and variability.
• Large Language Models (LLMs) show promise but have suboptimal performance in direct ICD code generation.

Purpose of the Study:

• To propose and evaluate a novel ICD coding paradigm using LLMs for enhanced code verification.
• To investigate the effectiveness of LLMs as code verifiers versus direct code generators.
• To assess the generalizability of LLM-based clinical coding systems.

Main Methods:

• Developed a new ICD coding paradigm focused on code verification using LLMs.
• Employed LLMs, including GPT-4o, to verify code assignments from candidate sets.
• Conducted extensive experiments on the CodiEsp dataset in zero-shot settings.

Main Results:

• LLMs perform more effectively as code verifiers than as direct code generators.
• GPT-4o achieved the best performance on the CodiEsp dataset under zero-shot conditions.
• LLM-based systems demonstrated performance comparable to state-of-the-art clinical coding systems.

Conclusions:

• LLMs are better suited for verifying clinical codes than generating them directly.
• The proposed code verification method improves generalizability across institutions, languages, and ICD versions.
• LLM-based clinical coding offers a robust alternative to existing systems.