Quantifying Evidence for Competing Biomedical Hypotheses using Large Language Models and Bayesian Analysis

Area of Science:

• Computational biology
• Scientific literature analysis
• Artificial intelligence in science

Background:

• Scientific progress relies on hypothesis testing, but literature growth complicates evaluation.
• Existing automated tools struggle to effectively compare competing hypotheses.
• Evaluating controversial scientific hypotheses is crucial yet challenging due to scale.

Purpose of the Study:

• To develop a transparent, reproducible algorithm for comparing controversial scientific hypotheses.
• To leverage large language models (LLMs) and co-occurrence methods for hypothesis evaluation.
• To provide a structured scoring approach with Bayesian confidence estimation.

Main Methods:

• Introduced KM-GPT-DCH, combining co-occurrence methods with LLMs.
• Developed a literature-based algorithm for hypothesis comparison.
• Utilized structured scoring and Bayesian methods for confidence assessment.

Main Results:

• KM-GPT-DCH accurately identified historical controversial hypotheses years ahead of scientific consensus.
• The algorithm demonstrated high confidence in selecting the correct hypothesis.
• Applied to 20 unresolved hypothesis pairs, offering guidance for future research.

Conclusions:

• KM-GPT-DCH offers a novel, effective method for evaluating and comparing scientific hypotheses.
• The algorithm can aid researchers and the public in understanding complex biomedical questions.
• Provides a tool for assessing and visualizing scientific literature trends.