Applying Large Language Models to Assess Quality of Care: Monitoring ADHD Medication Side Effects.
Yair Bannett1, Fatma Gunturkun2, Malvika Pillai3,4
1Division of Developmental-Behavioral Pediatrics, Stanford University School of Medicine, Stanford, California.
Pediatrics
|December 19, 2024
Summary
A large language model (LLM) accurately measured clinician adherence to ADHD medication guidelines by analyzing clinical notes. This technology offers scalable quality assessment for improving pediatric psychopharmacological care.
Area of Science:
- Artificial Intelligence in Healthcare
- Pediatric Psychopharmacology
- Clinical Quality Measurement
Background:
- Children with attention-deficit/hyperactivity disorder (ADHD) require careful medication management and monitoring for side effects.
- Assessing clinician adherence to practice guidelines for monitoring medication side effects in pediatric ADHD is crucial for quality care.
- Traditional methods of chart review for quality assessment are time-consuming and resource-intensive.
Purpose of the Study:
- To evaluate the accuracy of a large language model (LLM) in assessing clinician adherence to guidelines for monitoring ADHD medication side effects in children.
- To determine if an LLM can provide a scalable method for quality measurement in pediatric psychopharmacology.
Main Methods:
- A retrospective cohort study utilized electronic health records from 1201 children (aged 6-11 years) diagnosed with ADHD.
- An open-source LLM (LLaMA) was trained and deployed to analyze 15,628 clinical notes from ADHD-related encounters.
- Model performance was validated against manual medical record review, comparing its accuracy in identifying side effect inquiries.
Main Results:
- The LLM demonstrated high accuracy in identifying documentation of side effect inquiries (sensitivity 87.2%, specificity 86.3%, AUC 0.93).
- No significant model bias was found concerning patient sex or insurance status.
- Documented side effect inquiries were less frequent in telephone encounters (51.9%) compared to in-clinic/telehealth (73.0%) and varied between stimulant (61.4%) and nonstimulant (48.5%) prescriptions.
Conclusions:
- LLMs can be effectively deployed on diverse clinical notes, including telephone encounters, for scalable quality of care measurement.
- This approach facilitates the identification of areas for improvement in psychopharmacological medication management within primary care settings.
- Utilizing LLMs offers a promising avenue for enhancing the quality of care for children with ADHD.
More Related Videos
13:09Using Brain Activation nir-HEG/Q-EEG and Execution Measures CPTs in a ADHD Assessment Protocol
Published on: April 1, 2018
10.2K
05:48The Adventures of Fundi Intervention Based on the Cognitive and Emotional Processing in Attention Deficit Hyperactive Disorder Patients
Published on: June 12, 2020
5.7K
Related Concept Videos
01:28Drug Therapy
38
The advent of drug therapy has profoundly shaped modern mental health care, providing targeted treatments for a range of psychological disorders. Psychotherapeutic drugs, classified into antianxiety, antidepressant, and antipsychotic medications, address symptoms across anxiety disorders, mood disorders, and schizophrenia. While these medications have transformed patient outcomes, they require careful management due to their potential side effects and limitations.
Antianxiety Medications
Antianxiety Medications
38
01:26Modeling in Therapy
46
Modeling, a key technique in therapy, uses observational learning to help clients acquire and practice new skills by watching therapists demonstrate desired behaviors. This approach, rooted in Albert Bandura's concept of vicarious learning, plays a significant role in therapeutic interventions for various psychological conditions, including social anxiety, ADHD, and depression.
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
46
01:19Pharmacovigilance
773
Post-marketing surveillance is a critical component of pharmaceutical regulation, often uncovering unanticipated adverse drug reactions (ADRs) once a drug is widely used over an extended period.
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
773
01:30Attention-Deficit/Hyperactivity Disorder
38
Attention-deficit/hyperactivity disorder (ADHD) is a neurodevelopmental disorder characterized by persistent inattention, hyperactivity, and impulsivity. It affects approximately 5-8% of children globally, with around 60-70% of cases persisting into adulthood. ADHD has significant implications for educational attainment, social interactions, and occupational success.
Diagnostic Criteria and Symptoms
To diagnose ADHD, symptoms must manifest before age 12 and be evident across multiple settings....
Diagnostic Criteria and Symptoms
To diagnose ADHD, symptoms must manifest before age 12 and be evident across multiple settings....
38