Believing in black boxes: machine learning for healthcare does not need explainability to be evidence-based

Area of Science:

• Artificial Intelligence in Medicine
• Machine Learning for Healthcare (MLHC)

Background:

• The use of MLHC and artificial intelligence (AI) in medicine is rapidly expanding.
• Understanding the necessity and significance of explainability in MLHC is crucial for effective and ethical application.

Purpose of the Study:

• To examine the role, necessity, and significance of explainability in MLHC.
• To review arguments for and against explainability in MLHC.

Main Methods:

• Narrative review of literature on MLHC and AI in medicine.
• Analysis of arguments concerning explainability in MLHC.

Main Results:

• Concerns about explainability are not unique to MLHC, extending to established treatments and human judgment.
• An analogy is drawn between explainability in MLHC and mechanistic reasoning in evidence-based medicine.

Conclusions:

• The value of MLHC explainability is instrumental, serving performance and trust, not intrinsic.
• Advocates for robust empirical evaluation of complex algorithmic systems over uncompromising pursuit of explainability.