Decide less, communicate more: On the construct validity of end-to-end fact-checking in medicine

Area of Science:

• Medical Informatics
• Computational Linguistics
• Public Health Communication

Background:

• Automatic fact-checking has advanced, yet its application in medicine is limited.
• The complexity of medical literature challenges public health and evidence-based medicine accessibility.
• End-to-end fact-checking agents for medical claims remain largely unused.

Purpose of the Study:

• To examine how clinical experts verify real-world medical claims from social media.
• To identify fundamental challenges in applying end-to-end fact-checking to medical evidence synthesis.
• To propose an alternative approach to medical fact-checking.

Main Methods:

• Qualitative study involving clinical experts.
• Analysis of real-world social media claims and medical literature synthesis.
• Expert input used to develop the study's framework.

Main Results:

• Significant difficulties exist in linking social media claims to clinical trial evidence.
• Ambiguities in claims and subjective veracity labels pose major hurdles.
• Current end-to-end systems do not adequately address the nuances of medical fact-checking.

Conclusions:

• End-to-end fact-checking is insufficient for the medical domain.
• Medical fact-checking should be reframed as an interactive communication problem.
• Further research is needed to develop interactive systems for medical claim verification.