NLP modeling recommendations for restricted data availability in clinical settings.

Area of Science:

• Natural Language Processing (NLP)
• Clinical Informatics
• Computational Linguistics

Background:

• Healthcare decision-making heavily relies on unstructured clinical text.
• Traditional analysis methods struggle with clinical text data.
• Limited data and domain knowledge hinder clinical NLP adoption.

Purpose of the Study:

• Evaluate NLP modeling paradigms for Spanish clinical tasks.
• Assess model performance across varying data availability.
• Provide recommendations for clinical NLP practitioners.

Main Methods:

• Experimental analysis of NLP models on clinical tasks (referral prioritization, specialty classification).
• Simulated three clinical settings with different data availability levels.
• Evaluated four foundation models, including clinical PLMs and LLMs.

Main Results:

• Clinical PLMs achieved superior performance: 88.85% macro F1 for referral prioritization and 53.79% for specialty classification.
• Domain-specific pre-training improved performance, though gains were marginal relative to computational cost.
• Few-shot learning with LLMs showed promise in low-data scenarios.

Conclusions:

• Clinical-specific PLMs are most effective for clinical NLP tasks.
• Model selection should consider data availability, task complexity, and institutional readiness.
• Findings guide the development of practical clinical NLP solutions.