Related Experiment Videos
Effect of training and strict guidelines on the reliability of risk adjustment systems in paediatric intensive care
Jolanda G van Keulen1, Reinoud J B J Gemke, Kees H Polderman
1Department of Paediatrics, VU University Medical Center, P.O. Box 7057, 1007 MB, Amsterdam, The Netherlands.
Insights
Training significantly improved the reliability of the Paediatric Risk of Mortality (PRISM) and Paediatric Index of Mortality (PIM) scoring systems in pediatric intensive care units. This enhanced agreement is crucial for accurate quality benchmarking.
Area of Science:
- Pediatric Intensive Care
- Clinical Risk Adjustment
- Healthcare Quality Improvement
Background:
- Risk-adjustment systems in clinical practice often suffer from interobserver variability.
- This variability has not been extensively studied in the pediatric intensive care setting.
- Improving reliability is key for accurate quality assessment and benchmarking.
Purpose of the Study:
- To assess the reliability of the Paediatric Risk of Mortality (PRISM) and Paediatric Index of Mortality (PIM) scoring systems.
- To evaluate the impact of a specialized training program on the reliability of these pediatric scoring systems.
- To determine if training can reduce interobserver variability in daily clinical practice.
Main Methods:
- A prospective, observational, multi-centered cohort study was conducted.
- Twenty-seven physicians from five PICUs assessed 20 patients using PRISM and PIM scores.
- Physicians' scoring reliability was compared before and after a dedicated training program using intraclass correlation coefficients.
Main Results:
- Before training, significant interobserver variability was noted for both PRISM (ICC 0.24-0.73) and PIM (ICC 0.16-0.33) scores.
- Following training and guideline implementation, interobserver agreement substantially increased (PRISM ICC 0.74-0.86; PIM ICC 0.88-0.95).
- Despite improvements, some degree of scoring variability persisted post-training.
Conclusions:
- The reliability of PRISM and PIM scoring systems in routine pediatric intensive care is lower than anticipated.
- A structured training program and clear guidelines can significantly enhance interobserver agreement for these risk-adjustment tools.
- These findings underscore the importance of training and guidelines when utilizing PRISM and PIM for benchmarking and quality comparisons between PICUs.
Objective:
Many risk-adjustment systems have significant interobserver variability in everyday clinical practice. This can be partly corrected by strict guidelines and training. These issues have not been well studied in the paediatric setting. We assessed the reliability of two widely used paediatric scoring systems, the Paediatric Risk of Mortality (PRISM) and Paediatric Index of Mortality (PIM), before and after a special training program.
Design:
Prospective observational multi-centred cohort study.
Intervention:
Twenty-seven physicians from five paediatric intensive care units (PICUs) assessed severity of illness in 20 selected patients using PRISM and PIM scores before and after a special training program. Physicians were divided according to level of PICU experience: intensivists (>3 years experience, n=12), fellows (6-30 months experience, n=6) and residents (<6 months experience, n=9). Intraclass correlation was used to compare scoring reliability before and after training.
Measurements And Results:
Wide variability in PRISM and PIM scoring was observed before training (intraclass correlation for PRISM scores 0.24-0.73, intraclass correlation for PIM scores 0.16-0.33). Training and implementation of guidelines led to significant increases in interobserver agreement (intraclass correlation 0.74-0.86 for PRISM and 0.88-0.95 for PIM scores), although some variability remained.
Conclusion:
Our results show that the reliability of PRISM and PIM risk adjustment systems in daily clinical practice is much lower than expected. Training and guidelines can significantly increase interobserver agreement. These factors should be taken into account when using these systems for benchmarking, or to compare quality of care between different PICUs.
Related Concept Videos
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Pharmacokinetics in Pediatric Patients: Drug Distribution
Pharmacokinetics in Pediatric Patients: Overview and Drug Absorption