Clinical entity augmented retrieval for clinical information extraction.

Area of Science:

• Medical Informatics
• Artificial Intelligence
• Natural Language Processing

Background:

• Large language models (LLMs) with retrieval-augmented generation (RAG) offer advancements in information extraction.
• Current embedding-based RAG methods face inefficiencies in information retrieval.
• Accurate and efficient extraction of clinical data is crucial for healthcare.

Purpose of the Study:

• To introduce CLinical Entity Augmented Retrieval (CLEAR), a novel RAG pipeline.
• To evaluate CLEAR's performance against embedding RAG and full-note approaches for clinical data extraction.
• To assess CLEAR's efficiency in terms of inference time, model queries, and token usage.

Main Methods:

• Developed CLEAR, a RAG pipeline utilizing clinical entities for information retrieval.
• Compared CLEAR with embedding RAG and full-note methods across six LLMs.
• Extracted 18 variables from 20,000 clinical notes using the different approaches.

Main Results:

• CLEAR achieved an average F1 score of 0.90, outperforming embedding RAG (0.86) and full-note (0.79) approaches.
• CLEAR demonstrated significantly faster inference times (4.95s/note) compared to embedding RAG (17.41s/note) and full-note (20.08s/note).
• CLEAR reduced token usage by over 70% and model queries by over 65% compared to embedding RAG.

Conclusions:

• CLEAR effectively utilizes clinical entities for efficient and accurate information retrieval.
• CLEAR represents a substantial improvement over existing RAG methods for clinical data extraction.
• The CLEAR pipeline offers a more efficient and performant solution for processing clinical notes.