Could ChatGPT and co. replace forensic experts? A comparative study on medical liability expertise

Area of Science:

• Medical Law
• Artificial Intelligence
• Forensic Medicine

Background:

• Generative artificial intelligence (GAI) models have advanced significantly since 2022.
• Their potential application in specialized fields like forensic expertise is being explored.
• Medical liability assessments require high accuracy and rigorous reasoning.

Purpose of the Study:

• To evaluate the capability of current GAI models in conducting medical liability expertises.
• To compare GAI model performance against human expert conclusions in simulated malpractice cases.

Main Methods:

• Nine fictional medical malpractice cases were presented to three GAI models (ChatGPT-4 Turbo, Gemini, Mistral AI).
• Each model was queried five times per case to assess consistency.
• GAI conclusions were compared against a panel of three medical experts.

Main Results:

• GAI models' conclusions aligned with expert panels in 86 out of 135 instances.
• Discrepancies included missed shortcomings (false negatives) and unfounded deficiencies (false positives).
• All tested GAI models produced contradictory answers for the same case at least once.

Conclusions:

• Current GAI models exhibit risks of false negatives, false positives, and internal inconsistencies.
• The probabilistic nature of GAI is not currently suited for the rigorous demands of forensic expertise.
• GAI may serve as a future complement to human experts, but final decisions must remain with trained professionals.