Dictionary-Augmented Large Language Model Postprocessing for Bilingual Code-Switched Medical Speech Recognition:...

Clinical documentation burden contributes significantly to physician burnout, with health care professionals spending much of their time on electronic health record interactions. Automatic speech recognition (ASR) systems offer a promising solution; however, their application in Korean medical settings faces unique challenges due to widespread Korean-English code-switching, where clinicians routinely alternate between Korean conversational language and English medical terminology within single utterances.

This study aimed to develop and evaluate a hybrid postprocessing approach combining medical terminology dictionary normalization with large language model (LLM)-based postprocessing to improve ASR accuracy for Korean-English code-switched medical speech.

We constructed a speech dataset from 23,652 nursing progress notes, with a linguistic composition of 67.73% (512,626/756,866) Korean, 23.54% (178,166/756,866) English, and 8.73% (66,074/756,866) numerals or special symbols. Four Korean nurses recorded the notes using 5 microphone types in an acoustically isolated environment. Speech recognition was performed using OpenAI's gpt-4o-transcribe model. For postprocessing, a medical terminology dictionary containing 1070 mapping entries was constructed from 1000 nursing progress notes to normalize Korean phonetic renderings of English medical terms. Six LLMs (2 GPT and 4 Claude variants) were then evaluated across 5 temperature settings (0.0-0.8). Performance was assessed using BERTScore (bidirectional encoder representations from transformers score; F1), Sentence-BERT cosine similarity, word error rate, and character error rate (CER), comparing postprocessed outputs against the original written notes. Statistical significance was assessed using paired Wilcoxon signed-rank tests with Holm correction (α=.05).

Temperature optimization showed that all postprocessing models had small temperature-related effect sizes (all |Cohen dz| ≤0.15), with GPT-4o exhibiting the largest dependency and statistically significant improvement at temperature 0.6 (Holm-adjusted P<.001 for both BERTScore and CER) and the 4 Claude variants and GPT-4.1 exhibiting practically consistent performance across all settings. Baseline ASR achieved a BERTScore of 0.9131 and CER of 0.2336. Dictionary-based normalization performed 43,507 word-level substitutions in 70.8% (16,754/23,652) of transcribed sentences. LLM-only postprocessing reduced CER by 36.09% (Claude Sonnet 4) and 32.53% (GPT-4o) compared to baseline. The combined dictionary-LLM approach achieved the best performance: Claude Sonnet 4 attained a BERTScore of 0.9638 and CER of 0.0820, representing a 64.9% reduction in CER from baseline (P<.001).

The hybrid pipeline integrating rule-based dictionary normalization with LLM postprocessing significantly improved Korean-English code-switched medical ASR accuracy. Dictionary-based normalization yielded consistent additional improvements over LLM-only postprocessing for both GPT-4o and Claude Sonnet 4. As this modular framework requires no model retraining, it offers a practical means of mitigating multilingual challenges in medical ASR.