Integrating semantic retrieval and chain-of-thought reasoning in small language models for SNOMED CT normalization

Breast lesion biopsy assessment generates a high volume of pathology reports, posing a significant workload for pathologists. Standardized coding systems such as SNOMED CT Morphological codes enable consistent documentation, facilitate accurate data sharing, support clinical decision-making, and allow automated quality control. This study aims to evaluate systems that assist pathologists in normalizing and classifying free-text pathology reports to SNOMED CT Morphological codes, providing a short list of candidate codes for selection.

We used 2,718 breast biopsy pathology reports from over 20 hospitals, reported by nine expert pathologists in total. A normalization pipeline combining Small Language Models (SLMs) with semantic retrieval was evaluated to map free-text reports to SNOMED CT Morphological codes. Three strategies were evaluated: zero-shot prompting, Chain-of-Thought (CoT) with retrieval-augmented generation (RAG), and RAG combined with CoT, each generating a short list of candidate codes for pathologist selection. The strategies were assessed using ranking-oriented metrics adapted to the multi-label setting, including Hit@K, Mean Reciprocal Rank (MRR), Normalized Discounted Cumulative Gain (nDCG@K), and Recall@K, which measure both the presence and ranking of correct codes within the top-K predictions. Additionally, out-of-vocabulary (OOV) metrics were reported.

The RAG + CoT strategy achieved the highest performance, with Hit@5 scores of 70.97% for LLaMA and 72.11% for Gemma and demonstrated a strong concentration of correct codes at Rank 1. CoT + RAG improved over zero-shot prompting but tended to place correct codes at lower ranks.

Integrating SLMs with RAG and CoT provides an effective tool to support pathologists in coding breast biopsy pathology reports. By offering a short, curated list of SNOMED CT Morphological codes, the system enhances clinical workflow, improves data quality, and supports both prospective and retrospective analyses.