Evaluating the decision accuracy and speed of clinical data visualizations

Area of Science:

• Medical Informatics
• Biomedical Data Analysis
• Clinical Decision Support

Background:

• Clinicians are overwhelmed by increasing biomedical data volumes.
• Accurate and timely clinical decision-making systems require rigorous evaluation.
• Traditional methods for assessing system efficacy have limitations.

Purpose of the Study:

• To introduce and advocate for the use of multiple-reader multiple-case (MRMC) experimental design and linear mixed models.
• To demonstrate the utility of these methods for evaluating clinical decision support systems.
• To highlight the advantages of MRMC and linear mixed models over traditional quantitative methods.

Main Methods:

• Utilizes multiple-reader multiple-case (MRMC) experimental design.
• Employs linear mixed models for statistical analysis.
• Applies techniques extensively used in radiology imaging studies.

Main Results:

• MRMC and linear mixed models provide statistically efficient and generalizable assessments.
• These methods effectively evaluate both decision accuracy and latency (time).
• Offers practical and analytic advantages over traditional methods like percent-correct and ANOVAs.

Conclusions:

• MRMC experimental design and linear mixed models are valuable tools for medical informatics research.
• These techniques enhance the evaluation of clinical decision support systems.
• Recommended for assessing accuracy and latency in clinician decision tasks involving visual data interpretation.