Related Experiment Video
Updated: Aug 9, 2025
A Pediatric Concussion Model in Mice: Closed Head Injury with Long-Term Disorders (CHILD)
Published on: February 7, 2025
Predictability and stability testing to assess clinical decision instrument performance for children after blunt
Aaron E Kornblith1,2, Chandan Singh3, Gabriel Devlin2
1Department of Emergency Medicine, University of California, San Francisco, San Francisco, United States of America.
Insights
The Predictability Computability Stability (PCS) framework validated the Pediatric Emergency Care Applied Research Network (PECARN) clinical decision instrument (CDI) for identifying children with intra-abdominal injury. Three stable variables from the PECARN CDI maintained performance in external validation.
Area of Science:
- Emergency Medicine
- Pediatric Surgery
- Data Science in Healthcare
Background:
- The Pediatric Emergency Care Applied Research Network (PECARN) developed a clinical decision instrument (CDI) to identify children at very low risk of intra-abdominal injury.
- External validation of the PECARN CDI is crucial but has not been performed.
- The Predictability Computability Stability (PCS) data science framework can vet CDIs before external validation.
Purpose of the Study:
- To vet the PECARN CDI using the PCS data science framework.
- To assess the performance of the PECARN CDI and develop new PCS CDIs for external validation.
- To determine if the PCS framework can increase the success rate of external validation.
Main Methods:
- Secondary analysis of two prospectively collected datasets: PECARN (12,044 children) and Pediatric Surgical Research Collaborative (PedSRC; 2,188 children).
- Reanalysis of the original PECARN CDI and development of new PCS CDIs using the PECARN dataset.
- External validation of the CDIs on the independent PedSRC dataset.
Main Results:
- Three stable predictor variables identified: abdominal wall trauma, Glasgow Coma Scale Score <14, and abdominal tenderness.
- A CDI using these three variables achieved 96.8% sensitivity and 44% specificity on external validation, matching the original PECARN CDI's performance.
- The developed PCS CDI demonstrated equivalent performance to the original PECARN CDI on external validation.
Conclusions:
- The PCS framework successfully vetted the PECARN CDI and its predictor variables prior to external validation.
- Three stable predictor variables captured the PECARN CDI's full predictive performance on independent external validation.
- The PCS framework offers a less resource-intensive method to vet CDIs, potentially increasing the success of costly prospective external validation.
Objective:
The Pediatric Emergency Care Applied Research Network (PECARN) has developed a clinical-decision instrument (CDI) to identify children at very low risk of intra-abdominal injury. However, the CDI has not been externally validated. We sought to vet the PECARN CDI with the Predictability Computability Stability (PCS) data science framework, potentially increasing its chance of a successful external validation.
Materials & Methods:
We performed a secondary analysis of two prospectively collected datasets: PECARN (12,044 children from 20 emergency departments) and an independent external validation dataset from the Pediatric Surgical Research Collaborative (PedSRC; 2,188 children from 14 emergency departments). We used PCS to reanalyze the original PECARN CDI along with new interpretable PCS CDIs developed using the PECARN dataset. External validation was then measured on the PedSRC dataset.
Results:
Three predictor variables (abdominal wall trauma, Glasgow Coma Scale Score <14, and abdominal tenderness) were found to be stable. A CDI using only these three variables would achieve lower sensitivity than the original PECARN CDI with seven variables on internal PECARN validation but achieve the same performance on external PedSRC validation (sensitivity 96.8% and specificity 44%). Using only these variables, we developed a PCS CDI which had a lower sensitivity than the original PECARN CDI on internal PECARN validation but performed the same on external PedSRC validation (sensitivity 96.8% and specificity 44%).
Conclusion:
The PCS data science framework vetted the PECARN CDI and its constituent predictor variables prior to external validation. We found that the 3 stable predictor variables represented all of the PECARN CDI's predictive performance on independent external validation. The PCS framework offers a less resource-intensive method than prospective validation to vet CDIs before external validation. We also found that the PECARN CDI will generalize well to new populations and should be prospectively externally validated. The PCS framework offers a potential strategy to increase the chance of a successful (costly) prospective validation.
More Related Videos
08:13Development and Implementation of a Multi-Disciplinary Technology Enhanced Care Pathway for Youth and Adults with Concussion
Published on: January 20, 2019
08:07Author Spotlight: Advancing Traumatic Brain Injury Research - A Closed-Head Model for Accurate Replication and Rapid Assessment
Published on: September 22, 2023
