A Language Model for Pediatric Occupational Therapy Documentation: Model Development and Pilot Study.

Area of Science:

• Artificial Intelligence in Healthcare
• Clinical Documentation Improvement
• Occupational Therapy Practice

Background:

• Occupational therapy progress notes are crucial but time-consuming administrative tasks.
• Therapists dedicate significant time to documentation, contributing to patient waitlists.

Purpose of the Study:

• To develop a custom large language model (LLM) for efficient progress note generation in pediatric occupational therapy.
• To convert point-form notes into structured Subjective-Objective-Assessment-Plan (SOAP) documentation.

Main Methods:

• Training Llama 2 and 3 models using historical progress notes and synthetic scratch notes generated via few-shot prompting.
• Fine-tuning a Llama 3 8B Instruct model with domain-adaptive pretraining and low-rank adaptation.
• Piloting the custom LLM against Microsoft Copilot with ten therapists in a pediatric rehabilitation setting.

Main Results:

• Custom model notes demonstrated superior clarity, completeness, relevance, and organization compared to manual notes (P<.001).
• The custom model's notes were more concise than Copilot's (P<.001).
• No significant reduction in documentation time was observed in the pilot study without specific coaching.

Conclusions:

• The LLM can enhance progress note quality when therapists provide detailed input.
• Time savings were only realized when therapists used sparse input, a behavior they tended to abandon.
• The primary benefit observed was improved documentation quality rather than efficiency gains.