Related Experiment Video
Updated: Jun 23, 2026
Virtual Agent for Real-Time Motivational Interviewing by Integrating Adaptive Nonverbal Behavior and Language Models
Published on: December 23, 2025
Automating the roter interaction analysis system for medication counseling: A transformer-based deep learning
Ayako Mori1, Satoshi Watabe2, Izumi Kato3
1Education Research Center for Clinical Pharmacy, Faculty of Pharmaceutical Sciences, Hokkaido University, Kita 12, Nishi 6, Kita-ku, Sapporo, 060-0812, Japan; Laboratory of Clinical Pharmaceutics & Therapeutics, Division of Pharmasciences, Faculty of Pharmaceutical Sciences, Hokkaido University, Kita 12, Nishi 6, Kita-ku, Sapporo, 060-0812, Japan.
Background:
The Roter Interaction Analysis System (RIAS) is the gold standard for medical communication analysis, but its automated coding remains underexplored due to imbalanced code distributions. This study developed an automated RIAS classification system using Japanese transformer-based models and evaluated AI-based data augmentation to mitigate class imbalance.
Methods:
Five transformer models were fine-tuned for 44-class RIAS classification using medication counseling dialogues. To enhance generalizability, we employed AI-based data augmentation and evaluated performance using both AI-augmented and real-data-only test sets. Assessment metrics included accuracy, macro F1, and weighted F1.
Findings:
The dataset comprised 17,391 utterances (39.4% AI-generated). In the real-data-only (primary) test set, ELECTRA achieved the highest accuracy (0.7875), macro F1 (0.6561), and weighted F1 (0.7835). All models performed worse under the real-data-only condition than under the AI-augmented condition, mainly for minority-class categories. Error analysis showed semantically ambiguous and context-dependent categories, including domain-adjacent counseling topics and affective expressions, remained challenging, indicating linguistic transparency and definitional distinctiveness influence classification beyond training frequency.
Conclusion:
This proof of concept shows that AI-based augmentation can mitigate class imbalance in automated RIAS classification of medication counseling dialogues. While robust for prototypical expressions, nuanced affective categories remain challenging for text-only approaches. These findings support the feasibility of automated RIAS analysis for pharmacy education while suggesting that multimodal approaches are needed to capture subtle emotional dynamics. By complementing manual coding with quantitative insights, this system may support timelier and objective feedback and ultimately contribute to patient-centered care.