Related Experiment Video
Updated: Aug 4, 2025
Measuring the Functional Abilities of Children Aged 3-6 Years Old with Observational Methods and Computer Tools
Published on: June 20, 2020
Interrater reliability of chart-based assessment of functional impairment after pediatric injury using the functional
Caroline G Melhado1, Emily Kao, Jacqueline Hogan-Schlientz
1From the Division of Pediatric Surgery (C.G.M., E.K., C.Q.S., L.E., A.R.J.), UCSF Benioff Children's Hospitals; Department of Surgery (C.G.M., E.K., C.Q.S., L.E., A.R.J.), University of California San Francisco, San Francisco; Trauma Program (J.H.-S., D.C., A.R.J.), UCSF Benioff Children's Hospital Oakland, Oakland; Department of Epidemiology & Biostatistics (A.M.S.), University of California San Francisco, San Francisco, California; and Division of Trauma and Burn Surgery (R.S.B.), Children's National Hospital, Washington, District of Columbia.
Insights
Retrospective chart-based Functional Status Scale (FSS) assessment in pediatric trauma is reliable for quality improvement. Higher impairment levels and therapy use may increase rater disagreement.
Area of Science:
- Pediatric Trauma Care
- Quality Improvement Metrics
- Functional Outcome Assessment
Background:
- Functional impairment is a key outcome in pediatric trauma care.
- The Functional Status Scale (FSS) requires validation for retrospective chart-based assessment.
- Chart-based assessment offers a scalable alternative to in-person evaluations.
Purpose of the Study:
- To evaluate the interrater reliability of retrospective Functional Status Scale (FSS) assessment.
- To identify factors associated with unreliable retrospective FSS assessments in pediatric trauma patients.
Main Methods:
- Retrospective cohort analysis of pediatric trauma admissions (July 2020-June 2021).
- Two physicians and two nurse registrars assessed FSS domains from patient charts.
- Interrater reliability assessed using intraclass correlation (ICC); disagreement predictors analyzed via logistic regression.
Main Results:
- Excellent interrater reliability for total FSS score (ICC = 0.87) and good reliability for impairment categorization (ICC = 0.80).
- Rater disagreement occurred in 14% of cases; higher impairment and allied health therapy use were associated with disagreement.
- Median assessment time was 2 minutes per chart.
Conclusions:
- Retrospective chart-based FSS assessment is feasible and reliable for pediatric trauma.
- Detailed review may be needed for patients with higher functional impairment requiring therapy.
- Further validation is recommended prior to widespread implementation.
Background:
Functional impairment has been proposed as an alternative outcome for quality improvement in pediatric trauma. The functional status scale (FSS) has been used in studies of injured children, but has only been validated with resource-intensive in-person assessment. Implementation with retrospective chart-based FSS assessment would offer a simplified and scalable alternative. The purpose of this study was to evaluate interrater reliability of retrospective FSS assessment and to identify factors associated with unreliable assessment.
Methods:
A retrospective cohort of admissions to a Level I pediatric trauma center between July 2020 and June 2021 was analyzed. Two physicians and two nurse registrars reviewed charts to obtain measures of six FSS domains (mental status, sensory functioning, communication, motor functioning, feeding, and respiratory status) at discharge. Functional impairment was categorized by total FSS scores as good (6,7), mild impairment (8,9), moderate impairment (10-15), severe impairment (16-21), or very severe impairment (>21). Interrater reliability was assessed using intraclass correlation (ICC). Predictors of rater disagreement were evaluated using multivariable logistic regression.
Results:
The cohort included 443 children with a mean age of 7.4 years (standard deviation, 5.4 years) and median Injury Severity Score of 9 (interquartile range, 5-12). The median time per chart to assess FSS was 2 minutes (interquartile range, 1-2). Thirty-seven patients (8%) had functional impairment at discharge. Interrater reliability was excellent for total FSS score (ICC = 0.87) and good for FSS impairment categorization (ICC = 0.80). Rater disagreement of functional impairment categorization occurred in 14% of cases overall. Higher level of functional impairment and use of therapies (occupational and speech language therapy) were independently associated with more frequent rater disagreement.
Conclusion:
Chart-based FSS assessment is feasible and reliable, but may require more detailed review for patients with higher level of impairment that require allied health therapy. Validation of chart-based assessment is needed before widespread implementation.
Level Of Evidence:
Prognostic/Epidemiological, Level III.
