Project IMPACT: results from a pilot validity study of a new observational database

Area of Science:

• Healthcare Informatics
• Clinical Data Management
• Intensive Care Unit (ICU) Performance Measurement

Background:

• Project IMPACT is a database system used by hospitals to measure and improve intensive care patient care.
• Data entered includes patient demographics, diagnoses, treatments, and outcomes.
• Assessing data accuracy is crucial for the database's utility in performance measurement.

Purpose of the Study:

• To evaluate the accuracy of data within the Project IMPACT database.
• To determine the usefulness of Project IMPACT for measuring ICU performance and patient outcomes.

Main Methods:

• A pilot study involved independent data abstraction from random samples of ICU patient records at two hospitals.
• Data from patient records were compared against existing data in the Project IMPACT database.
• Abstraction was conducted by an unaffiliated, trained abstractor onsite at each hospital.

Main Results:

• Good agreement (≥80%) was observed for discrete data items, such as patient type.
• Poorer agreement (<80%) was noted for continuous data (e.g., urine output) and coded items requiring judgment (e.g., admission reason).
• Agreement rates were high regardless of data capture method (electronic vs. manual), with higher agreement for electronically downloaded data.

Conclusions:

• The Project IMPACT database demonstrates good internal validity for most abstracted variables.
• The database is a valuable resource for ICUs to collect and evaluate treatment and patient outcome information.
• Electronic data capture enhances the accuracy of data within the Project IMPACT system.