Evaluating Encoder and Decoder Models for Extended Clinical Concept Recognition in Japanese Clinical Texts:...

Area of Science:

• Natural Language Processing
• Biomedical Informatics
• Machine Learning

Background:

• Extracting medical knowledge for secondary uses like diagnostic support is challenging.
• Conventional methods focus on short terms, neglecting longer, clinically vital phrases (e.g., diseases, symptoms).
• Accurate extraction of long phrases is essential for applications like building causal knowledge from case reports.

Purpose of the Study:

• To investigate optimal strategies for extended Clinical Concept Recognition (E-CCR) model selection.
• To compare encoder vs. decoder models and general-purpose vs. domain-specific pretraining.
• To propose a novel E-CCR evaluation metric and analyze effectiveness by target length.

Main Methods:

• Evaluated 17 encoder and decoder models on the J-CaseMap Japanese case report database.
• Utilized a weighted soft matching score to penalize fragmentation and account for target length.
• Assessed performance using F1-scores and analyzed trends with increasing fragmentation penalties.

Main Results:

• The domain-specific encoder model JMedDeBERTa(s) achieved the highest mean performance (F1=0.758) on J-CaseMap.
• Encoder models generally outperformed decoder models, with token classification showing better results than instruction tuning for long expressions.
• Limited benefit of domain-specific pretraining was observed on a general-domain dataset.

Conclusions:

• Encoder-based token classification models offer high accuracy with fewer parameters, suitable for resource-constrained environments.
• Token classification is more effective for long expression extraction than instruction tuning.
• Findings suggest generalizability to Japanese medical text information extraction, warranting further cross-lingual and cross-document type investigation.