Developing a Large Language Model-Based Feedback System for Case Report Writing in Rehabilitation Education: Tutorial

Novice health care staff often write case reports during early clinical training. However, many institutions lack structured feedback systems because of time constraints and instructor shortages. Large language models, a form of artificial intelligence (AI), offer new opportunities to enhance educational feedback, yet their application in clinical training requires careful design to ensure pedagogically appropriate and ethically sound outputs.

This tutorial provides a practical guide for educators without programming experience to develop an AI-based feedback system using 3 accessible tools: Dify (an AI workflow platform), Slack (a messaging app), and Google Apps Script. The system balances educational quality with operational efficiency while incorporating data privacy safeguards for clinical educational content.

We developed a feedback system comprising 4 AI chatbots with 2 distinct approaches: "loop-based" bots that promote clinical reasoning through iterative, comment-based feedback and "single-shot" bots for efficient proofreading and summarization tasks. The tutorial describes the system architecture; feedback design principles grounded in formative assessment theory; a step-by-step implementation guide; and data privacy safeguards, including a deidentification protocol and application programming interface-based data protection measures. To illustrate feasibility, we conducted a pilot implementation at a community care hospital from April to June 2024, involving 5 novice staff members and 5 instructors.

A pilot implementation at a community care hospital demonstrated that the system was feasible to deploy and operate within routine clinical education workflows. Participant feedback indicated high usability and suggested that the iterative, comment-based feedback approach supported learner engagement while also identifying areas where feedback criteria required refinement to better match institutional educational goals.

This tutorial provides a reproducible framework for building a customized AI feedback system that combines comment-based iterative feedback with human-in-the-loop oversight and a multilayered data privacy framework. By following this guide, educators can implement an adaptive system tailored to their institutional context and clinical domain.