Capturing Requirements for a Data Annotation Tool for Intensive Care: Experimental User-Centered Design Study.

Area of Science:

• Clinical Informatics
• Machine Learning in Healthcare
• Data Annotation

Background:

• Computational methods and machine learning offer potential in healthcare but face challenges in data annotation.
• Data annotation requires domain expertise and time, resources clinicians often lack.
• Current tools are inadequate for clinical data annotation workflows.

Purpose of the Study:

• Investigate how intensive care unit (ICU) staff approach data annotation.
• Identify requirements for a digital data annotation tool tailored for healthcare settings.

Main Methods:

• An experimental activity involved 7 ICU clinicians annotating time-series admission data.
• Participants annotated periods of weaning from mechanical ventilation during a 45-minute workshop.
• Clinician actions were analyzed using Norman's Interaction Cycle to define software requirements.

Main Results:

• Clinicians utilized a cyclic process: investigation, annotation, reevaluation, and refinement.
• Eleven requirements for a digital tool were identified across four domains: individual admission annotation, semi-automated annotation, operational constraints, and machine learning label usage.
• Specific needs included flexibility in analysis and label creation, and workflow continuity.

Conclusions:

• Clinical data annotation success hinges on flexible analysis and label creation.
• Workflow continuity across multiple admissions is crucial for efficient annotation.
• A seamless transition between data investigation, annotation, and label refinement is necessary.