Developing ICU Clinical Behavioral Atlas Using Ambient Intelligence and Computer Vision

Area of Science:

• Medical Technology
• Computer Vision
• Artificial Intelligence in Healthcare

Background:

• Limited availability of computer vision models for intensive care unit (ICU) activities.
• Growing interest in leveraging AI for medical applications.

Purpose of the Study:

• To develop and evaluate a computer vision system, Clinical Behavioral Atlas (CBA), for identifying clinically relevant activities and objects in the ICU.
• To assess the performance of CBA using a large-scale dataset of ICU video recordings.

Main Methods:

• Development of CBA using over 140,000 hours of RGB video data from 16 sensors in 8 ICU rooms.
• Annotation of over 350,000 frames to identify 40 activity and 55 object categories.
• Evaluation of model performance using sensitivity, average precision, and permutation tests.

Main Results:

• CBA achieved strong performance in entity and activity detection (sensitivities 0.75-0.81, average precisions 0.64-0.73).
• Model performance correlated positively with entity number/size and video duration.
• CBA significantly outperformed existing activity recognition models, demonstrating its potential for clinical applications.

Conclusions:

• CBA demonstrates the feasibility of using computer vision to automate the identification of critical bedside clinical actions in the ICU.
• Further development with larger, multi-location datasets is needed for full clinical-level performance.
• The system can assist in monitoring ICU preventive bundle elements and other clinical actions.