Related Experiment Video
Updated: Feb 15, 2026

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity
Published on: June 12, 2019
Inter-rater reliability in the Paediatric Observation Priority Score (POPS)
Lisa Langton1, Adam Bonfield1, Damian Roland1,2
1Paediatric Emergency Medicine Leicester Academic (PEMLA) Group, University Hospitals of Leicester NHS Trust, Leicester, UK.
Insights
This study found good inter-rater reliability for the Paediatric Observation Priority Score (POPS) among nursing staff in emergency departments. The POPS tool effectively supports consistent assessment of pediatric patients.
Area of Science:
- Pediatric Emergency Medicine
- Nursing Assessment Tools
- Clinical Reliability Studies
Background:
- Accurate and consistent patient assessment is crucial in pediatric emergency settings.
- The Paediatric Observation Priority Score (POPS) is a tool used to prioritize pediatric patients.
- Evaluating the reliability of assessment tools ensures standardized care.
Purpose of the Study:
- To assess the inter-rater reliability of the Paediatric Observation Priority Score (POPS) among nursing staff.
- To determine the consistency of POPS scoring by different nurses in a pediatric emergency department.
Main Methods:
- A retrospective observational study was conducted in a single-center pediatric emergency department.
- 12 nursing staff participants reviewed video footage of pediatric assessments and scored them using POPS.
- Participants were blinded to original scores, and a questionnaire assessed their POPS experience.
Main Results:
- Inter-rater reliability for POPS was evaluated using kappa values.
- Kappa values were 0.74 for case 1, 1.00 for case 2 (perfect agreement), and 0.66 for case 3.
- These results indicate good to perfect agreement among raters.
Conclusions:
- The Paediatric Observation Priority Score (POPS) demonstrates good inter-rater reliability among nursing staff.
- POPS can be reliably used for assessing sick children in the emergency department.
- Consistent application of POPS supports standardized pediatric emergency care.
Objective:
The primary objective of this study was to determine the level of inter-rater reliability between nursing staff for the Paediatric Observation Priority Score (POPS).
Design:
Retrospective observational study.
Setting:
Single-centre paediatric emergency department.
Participants:
12 participants from a convenience sample of 21 nursing staff.
Interventions:
Participants were shown video footage of three pre-recorded paediatric assessments and asked to record their own POPS for each child. The participants were blinded to the original, in-person POPS. Further data were gathered in the form of a questionnaire to determine the level of training and experience the candidate had using the POPS score prior to undertaking this study.
Main Outcome Measures:
Inter-rater reliability among participants scoring of the POPS.
Results:
Overall kappa value for case 1 was 0.74 (95% CI 0.605 to 0.865), case 2 was 1 (perfect agreement) and case 3 was 0.66 (95% CI 0.58 to 0.744).
Conclusion:
This study suggests there is good inter-rater reliability between different nurses' use of POPS in assessing sick children in the emergency department.
More Related Videos
Related Concept Videos
Naturalistic Observations
Reliability and Validity
Introduction to z Scores
z scores...
Introduction to z Scores
z scores...
z Scores and Area Under the Curve
Distribution Reliability and Automation

