Related Experiment Videos
The evaluation of artificial intelligence systems in medicine
Computer Methods and Programs in Biomedicine
|March 1, 1986
Summary
Evaluating artificial intelligence in medicine requires addressing challenges at three levels: prototype research, system validation, and clinical efficacy. This framework helps assess AI
Area of Science:
- Medical Informatics
- Computer Science
- Artificial Intelligence
Background:
- Evaluating artificial intelligence (AI) in medicine presents unique challenges.
- Existing evaluation methods often lack a structured framework.
- Standardized approaches are needed for reliable AI system assessment.
Purpose of the Study:
- To identify and discuss key issues in evaluating computer systems using AI in medical applications.
- To propose a multi-level framework for AI in medicine evaluation.
- To contextualize previous AI in medicine evaluations within this framework.
Main Methods:
- Discussion of underlying issues in AI in medicine evaluation.
- Description of three distinct evaluation levels: prototype, system validation, and clinical efficacy.
- Analysis of evaluation challenges at each level.
Main Results:
- Identified subjective evaluation issues for developmental prototypes.
- Outlined knowledge and performance validation challenges for AI systems.
- Discussed clinical efficacy evaluation issues for operational AI systems.
Conclusions:
- A structured, multi-level approach is crucial for evaluating AI in medicine.
- Understanding evaluation issues at each level enhances assessment rigor.
- The proposed framework provides a basis for future AI in medicine evaluations.