Korean Large Language Models for Medical Question Answering on Arthritis: Fine-tuning and Comparative Evaluation.

Area of Science:

• Artificial Intelligence in Medicine
• Natural Language Processing
• Clinical Informatics

Background:

• Large language models (LLMs) are increasingly used for medical question-answering (QA).
• English-centric LLM training limits effectiveness in non-English clinical settings, like Korea.
• There is a need for LLMs adapted to local languages and medical domains.

Purpose of the Study:

• Evaluate Korean-native and multilingual LLMs fine-tuned on Korean arthritis medical QA data.
• Assess the impact of language and domain adaptation on LLM performance.
• Compare different LLMs for their suitability in Korean clinical contexts.

Main Methods:

• Constructed a dataset of 5,451 Korean arthritis QA pairs.
• Fine-tuned five LLMs (Mi:dm, EXAONE, Kanana, HyperCLOVAX, LLaMA) using Low-Rank Adaptation with 4-bit quantization.
• Evaluated models on 597 validation samples using quantitative (BERTScore-F1, SBERT similarity) and qualitative metrics (clinical correctness, safety, completeness).

Main Results:

• EXAONE and HyperCLOVAX demonstrated strong quantitative performance in semantic accuracy and consistency.
• Mi:dm excelled in qualitative evaluation, showing superior clinical correctness and safety.
• Kanana had moderate performance with limited domain adaptability; LLaMA showed the lowest Korean medical QA performance despite relative gains.

Conclusions:

• Domain-specific fine-tuning and Korean-oriented design enhance LLM performance for Korean medical QA.
• EXAONE and HyperCLOVAX lead in semantic similarity, while Mi:dm leads in clinical correctness and safety.
• Developing disease-specific Korean medical LLMs is crucial, as general multilingual models show limitations.