Related Experiment Video
Updated: Aug 19, 2026
Multimedia Battery for Assessment of Cognitive and Basic Skills in Mathematics (BM-PROMA)
Published on: August 28, 2021
Reliability of PRISM and PIM scores in paediatric intensive care
J G van Keulen1, K H Polderman, R J B J Gemke
1Department of Pediatrics, VU University Medical Center, Amsterdam, Netherlands.
Insights
Mortality risk assessment using the Paediatric Risk of Mortality (PRISM) score and Paediatric Index of Mortality (PIM) shows significant variability in daily practice. Physician experience did not explain these differences, highlighting the need for standardized training.
Area of Science:
- Pediatric Intensive Care
- Clinical Outcomes Research
- Healthcare Quality Improvement
Background:
- Accurate mortality risk assessment is crucial in pediatric intensive care units (PICUs).
- The Paediatric Risk of Mortality (PRISM) score and Paediatric Index of Mortality (PIM) are widely used for risk adjustment.
- Variability in scoring can impact patient management and resource allocation.
Purpose of the Study:
- To evaluate the reliability of the PRISM and PIM scores in routine clinical practice.
- To determine if physician experience influences the consistency of these risk assessment tools.
Main Methods:
- Twenty-seven physicians from eight tertiary PICUs assessed 10 representative patients using PRISM and PIM scores.
- Physicians were categorized by experience: intensivists, fellows, and residents.
- Intraclass correlation (ICC) and kappa scores were used to measure inter-rater reliability.
Main Results:
- Significant variability was observed in both PRISM and PIM scores among physicians.
- Average ICC for PRISM was 0.51, and for PIM was 0.18, indicating moderate to poor reliability.
- Kappa scores also showed considerable variation, with no clear trend related to physician experience.
- Exact agreement percentages ranged widely for both scoring systems.
Conclusions:
- The application of PRISM and PIM scores in daily practice is subject to substantial variability.
- Physician experience level does not adequately explain the observed inconsistencies in scoring.
- Standardized training and strict adherence to guidelines are essential for reliable PRISM and PIM assessments.
- Consideration should be given to limiting the number of trained professionals performing these assessments.
Aims:
To assess the reliability of mortality risk assessment using the Paediatric Risk of Mortality (PRISM) score and the Paediatric Index of Mortality (PIM) in daily practice.
Methods:
Twenty seven physicians from eight tertiary paediatric intensive care units (PICUs) were asked to assess the severity of illness of 10 representative patients using the PRISM and PIM scores. Physicians were divided into three levels of experience: intensivists (>3 years PICU experience, n = 12), PICU fellows (6-30 months of PICU experience, n = 6), and residents (<6 months PICU experience, n = 9). This represents all large PICUs and about half of the paediatric intensivists and PICU fellows working in the Netherlands.
Results:
Individual scores and predicted mortality risks for each patient varied widely. For PRISM scores the average intraclass correlation (ICC) was 0.51 (range 0.32-0.78), and the average kappa score 0.6 (range 0.28-0.87). For PIM scores the average ICC was 0.18 (range 0.08-0.46) and the average kappa score 0.53 (range 0.32-0.88). This variability occurred in both experienced and inexperienced physicians. The percentage of exact agreement ranged from 30% to 82% for PRISM scores and from 28 to 84% for PIM scores.
Conclusion:
In daily practice severity of illness scoring using the PRISM and PIM risk adjustment systems is associated with wide variability. These differences could not be explained by the physician's level of experience. Reliable assessment of PRISM and PIM scores requires rigorous specific training and strict adherence to guidelines. Consequently, assessment should probably be performed by a limited number of well trained professionals.