Optimizing Agreement Between Bedside Nurse-Documented and Trained Researcher Delirium Assessments in the ICU

Delirium is common and harmful in the ICU. The Intensive Care Delirium Screening Checklist (ICDSC) and Confusion Assessment Method for the ICU (CAM-ICU) are validated tools recommended for delirium identification. However, the accuracy of bedside nurse-documented delirium assessments in the ICU is inconsistent, limiting utility in clinical research. We sought to evaluate and optimize agreement between bedside nurse-documented and trained researcher delirium assessments.

Critically ill adults with acute respiratory failure or sepsis in ICUs in large academic hospitals in a southwestern Pennsylvania health system were assessed daily for delirium by bedside nurses (using the ICDSC) and trained researchers (using the CAM-ICU). Using matched nurse-to-researcher delirium assessments, we categorized delirium status using validated cutoffs and evaluated agreement using Cohen's kappa. We derived and compared logistic regression models that used ICDSC documentation, mechanical ventilation status, and admission Sequential Organ Failure Assessment to predict delirium in noncomatose patients, using researcher CAM-ICU assessments as the reference standard. We internally validated models using ten-fold cross-validation.

None.

From a sample of 1535 matched assessments of 279 patients, there was moderate agreement between bedside nurse assessments using the established ICDSC delirium/normal cutoff (ICDSC ≥ 4) and trained researcher assessments using the CAM-ICU (Cohen's kappa = 0.42). A logistic regression model informed by individual ICDSC components and clinical data predicted a positive research CAM-ICU with good discrimination (area under the curve = 0.87) and performed well in cross-validation (F1 score = 0.72). In sensitivity analyses, models with more limited ICDSC information demonstrated fair to good discriminatory ability (F1 = 0.60-0.70), with the validated cutoff model having the lowest performance.

A delirium model informed by bedside nurse ICDSC findings and clinical variables improves accuracy of delirium detected in the ICU and can be used in future pragmatic research that leverages large clinical datasets to advance understanding of delirium mechanisms, trajectories, and outcomes.