An evaluation model for clinical information systems: clinical imaging systems

Area of Science:

• Health Informatics
• Information Systems Evaluation
• Clinical Computing

Background:

• Evaluating complex clinical information systems, such as integrated clinical imaging systems, presents significant challenges.
• Existing evaluation methods may not adequately address the multifaceted nature of these systems.

Purpose of the Study:

• To propose a comprehensive evaluation framework for complex clinical computer information systems.
• To outline a model evaluation plan applicable to clinical imaging systems and other health IT.

Main Methods:

• Developed a five-criterion model for comprehensive evaluation: technical, economic, organizational concerns; multiple methods; longitudinal; formative and summative; modifiable.
• Designed a three-phase model evaluation plan: 1) identifying research questions via literature synthesis and ethnographic methods, 2) documenting system use through critical incident reporting and user profiles/surveys, 3) evaluating the system in context across various clinical settings.
• Employed interviews, observations, literature synthesis, critical incident reporting, user profiles, and surveys.

Main Results:

• The proposed model and plan meet the five criteria for comprehensive evaluation.
• The plan allows for longitudinal and formative evaluations, incorporating feedback for system refinement.
• A preliminary evaluation at a U.S. Department of Veterans Affairs medical center identified clinical benefits in patient care, education, research, and administration, translating to cost savings and productivity improvements.

Conclusions:

• The proposed comprehensive evaluation plan provides a robust framework for assessing complex clinical information systems.
• The model is adaptable and can be modified to suit different systems and contexts.
• Effective evaluation leads to identification of tangible benefits, including improved patient care and operational efficiencies.