Evaluating Encoder and Decoder Models for Extended Clinical Concept Recognition in Japanese Clinical Texts: A...

Area of Science:

• Natural Language Processing
• Medical Informatics
• Computational Linguistics

Background:

• Digitized medical documents offer vast data, but extracting complex clinical concepts remains challenging.
• Conventional named entity recognition (NER) struggles with long phrases vital for applications like diagnostic support.
• Extended Clinical Concept Recognition (E-CCR) is essential for capturing these complex medical expressions.

Purpose of the Study:

• To identify optimal strategies for E-CCR model selection.
• To compare encoder vs. decoder models and general-purpose vs. domain-specific pretraining.
• To analyze model effectiveness based on target length and propose a novel evaluation metric.

Main Methods:

• Evaluated 17 encoder and decoder models on the J-CaseMap database (approx. 20,000 Japanese case reports).
• Utilized a novel "weighted soft matching score" to penalize fragmentation and weight by target length.
• Assessed performance variations concerning target length and pretraining strategies.

Main Results:

• Encoder models outperformed decoder models in the E-CCR task.
• JMedDeBERTa(s), an encoder model, achieved the highest mean performance (F1 = 0.758).
• Domain-specific pretraining showed limited benefits; token classification outperformed instruction tuning for long expressions.

Conclusions:

• Encoder-based token classification is highly effective for E-CCR, offering potential advantages in resource-constrained settings.
• Model performance was robust against fragmentation, indicating reliable extraction of long phrases.
• Findings suggest generalizability to Japanese medical text information extraction, warranting further cross-lingual and cross-document type investigation.