Developing a Novel Coding System for Retrospective Chart Review of Attention-Deficit/Hyperactivity Disorder, Anxiety,...

Pediatric mental health prevalence rates have increased in recent years, while gaps remain in the number of available providers. Ongoing evaluation and understanding of treatment progress and engagement are critical to psychiatric care, and these details are often documented in the electronic health record (EHR). Given the utility of retrospective chart review (RCR) as a tool for psychiatrists, we developed a coding system examining common comorbid conditions (anxiety and attention-deficit/hyperactivity disorder [ADHD]) and adherence and evaluated interrater reliability.

We created a coding system with a comprehensive manual and coding instructions that explore both symptom severity domains (anxiety, ADHD, and global) and adherence to medication. Codes were rated using Likert scales, and two independent raters coded all data.

RCR was completed for 142 patients with a total of 1139 visits over 2 years. Weighted linear kappa statistics ranged between 0.77 and 0.95, and weighted quadratic kappa statistics ranged between 0.74 and 0.96, suggesting substantial to almost perfect agreement. Interrater agreement was highest for anxiety severity.

We created a novel coding system for RCR and found substantial to almost perfect interrater reliability for assessing ADHD severity, anxiety severity, global severity, and medication adherence using psychiatry encounter notes documented in an EHR. Our coding system explores conditions that are often heterogeneous and have waxing and waning presentations, using a continuum that captures the complexity of symptoms. Future directions include utilization of coding systems to explore emotion and behavior change over time to optimize treatment.