Related Experiment Video
Updated: Jul 18, 2025
Evaluation of Capnography Sampling Line Compatibility and Accuracy when Used with a Portable Capnography Monitor
Published on: September 29, 2020
Interrater agreement in classifying infections during extracorporeal membrane oxygenation
Karlijn Verkerk1, Lara Ca Pladet1, Christiaan L Meuwese2,3
1Department of Intensive Care Medicine, University Medical Center Utrecht, Utrecht, The Netherlands.
Abstract:
Infectious complications are common during extracorporeal membrane oxygenation (ECMO) and may negatively impact outcomes. However, there is considerable variation in the reported rates of incidence, which hampers the use of infections as a quality benchmark for ECMO centers. To assess the contributing role of poor interrater agreement, three independent raters reviewed medical records from all intensive care unit (ICU) patients who received ECMO for >24 h in our tertiary center between October 2019 and October 2021 for suspected episodes of infection, which were rated based on their date of onset and presumed site/diagnosis. To establish a gold standard, any discrepancies were resolved using an expert panel consisting of two intensivists/infectious disease specialists. During 83 ECMO-runs in 77 patients, we observed a total of 62 adjudicated infectious episodes (incidence rate 62, 95% CI: 48-80, per 1000 days at risk). Among 81 episodes suspected by at least one observer, 66 (81%) were identified by two, and only 44 (54%) by all three raters, resulting in Fleiss' kappa of 0.10 (95% CI: 0.00-0.19; slight agreement). However, if raters concurred regarding infection onset, subsequent agreement on infection site was good (concordance 89%; kappa 0.85, 95% CI: 0.72-0.98; near perfect agreement). In conclusion, adjudication of infectious episodes during ECMO is associated with poor interrater agreement regarding occurrence-but not site-of infection. This finding might partially explain the significant disparities observed in reported infection rates during ECMO, emphasizing the need for caution when interpreting infection data in this particular population due to the potential for inherent measurement error.
Insights
Infectious episodes during extracorporeal membrane oxygenation (ECMO) show poor agreement among raters regarding occurrence, but good agreement on infection site. This variability may explain differing infection rates reported by ECMO centers.
Area of Science:
- Critical Care Medicine
- Infectious Diseases
- Medical Quality Improvement
Background:
- Infectious complications are frequent during extracorporeal membrane oxygenation (ECMO), impacting patient outcomes.
- Variability in reported infection incidence rates hinders the use of infections as a quality benchmark for ECMO centers.
Purpose of the Study:
- To evaluate the impact of interrater agreement on the assessment of infectious episodes in intensive care unit (ICU) patients undergoing ECMO.
- To determine if poor agreement in identifying infection occurrence or site contributes to discrepancies in reported ECMO-related infection rates.
Main Methods:
- Three independent raters reviewed medical records of ICU patients receiving ECMO for >24 hours to identify and date suspected infectious episodes.
- Discrepancies in infection identification, onset, and site were resolved by an expert panel of intensivists and infectious disease specialists.
- Interrater agreement was quantified using Fleiss' kappa statistic and concordance rates.
Main Results:
- A total of 62 infectious episodes were adjudicated across 83 ECMO runs in 77 patients (incidence rate: 62 per 1000 patient-days at risk).
- Only 54% of suspected infectious episodes were identified by all three raters (Fleiss' kappa = 0.10, indicating slight agreement).
- Agreement on infection site was high (89% concordance, kappa = 0.85, near perfect agreement) when raters agreed on infection onset.
Conclusions:
- Adjudication of infectious episodes during ECMO is characterized by poor interrater agreement on infection occurrence, but good agreement on infection site.
- This lack of consensus on infection occurrence may partially explain the wide disparities in reported infection rates across ECMO centers.
- Caution is advised when interpreting infection data in ECMO populations due to potential measurement errors stemming from interrater variability.
More Related Videos
Related Concept Videos
01:16Oxygen Delivering System II: Venturi Mask and Transtracheal Oxygen
Venturi Mask
The Venturi mask, named after the Venturi effect, is designed to deliver precise oxygen concentrations. It consists of a large tube with an oxygen inlet that narrows down, causing a pressure drop that pulls air in through adjustable side ports. The mask is a lightweight,...
01:29Pulmonary Embolism II: Diagnostic Studies and Interprofessional Care
01:23Mechanical Ventilation II: Invasive Ventilation
Negative-Pressure Ventilators
Negative-pressure ventilators create a vacuum around the chest or body to draw air into the lungs, simulating breathing. This method does not require an...
01:30Pneumonia III: Complications and Assessment
01:17Assessment of Diffusion and Perfusion
The Role of Diffusion in Respiration
Diffusion is the process by which molecules move from an area of higher concentration to an area of lower concentration. In the respiratory system, this...
01:21Acute Respiratory Failure-II
The underlying physiological abnormalities that contribute to hypoxemic respiratory failure include: