Integrating semantic retrieval and chain-of-thought reasoning in small language models for SNOMED CT normalization

Area of Science:

• Medical Informatics
• Computational Pathology
• Natural Language Processing

Background:

• Pathology reports for breast biopsies present a significant workload.
• Standardized coding systems like SNOMED CT Morphological codes are crucial for data consistency and quality control.

Purpose of the Study:

• To evaluate systems that assist pathologists in normalizing free-text pathology reports to SNOMED CT Morphological codes.
• To provide a short list of candidate codes for efficient pathologist selection.

Main Methods:

• Utilized 2,718 breast biopsy pathology reports.
• Developed a normalization pipeline combining Small Language Models (SLMs) with semantic retrieval.
• Evaluated three strategies: zero-shot prompting, Chain-of-Thought (CoT) with retrieval-augmented generation (RAG), and RAG combined with CoT.

Main Results:

• The RAG + CoT strategy demonstrated superior performance, achieving high Hit@5 scores (e.g., 72.11% with Gemma).
• This strategy showed a strong concentration of correct codes at Rank 1.
• CoT + RAG outperformed zero-shot prompting but ranked correct codes lower than RAG + CoT.

Conclusions:

• Integrating SLMs with RAG and CoT offers an effective tool for coding breast biopsy reports.
• The system enhances clinical workflow and data quality by presenting a curated list of SNOMED CT codes.
• This facilitates both prospective and retrospective analyses.