EA3: A softmax algorithm for evidence appraisal aggregation

Area of Science:

• Medical research methodology
• Health informatics
• Biostatistics

Background:

• Real World Evidence (RWE) is increasingly utilized in medical research and regulatory decision-making, notably for risk-benefit assessments under the 21st Century Cures Act.
• Assessing the quality and inferential strength of RWE is challenging due to potential imperfections in evidence production methodologies.
• Aggregating multiple appraised imperfections to perform robust inference with RWE presents a significant methodological hurdle.

Purpose of the Study:

• To develop a robust algorithm for appraising and aggregating Real World Evidence (RWE).
• To enhance the inferential strength of RWE for medical interventions.
• To support causal inference within a Bayesian decision-making framework using aggregated RWE appraisals.

Main Methods:

• Development of the Evidence Appraisal Aggregation Algorithm (EA3).
• Utilization of the softmax function, a generalized logistic function, for evidence aggregation.
• Demonstration of EA3's properties and application in supporting causal inferences via a Bayesian decision-making framework.

Main Results:

• EA3 demonstrates desirable properties for appraising Real World Evidence (RWE).
• Aggregated evidence appraisals from EA3 effectively support causal inferences.
• The algorithm provides a framework for handling imperfections in RWE methodologies.

Conclusions:

• The EA3 algorithm offers a promising approach to address challenges in RWE quality appraisal and aggregation.
• EA3 facilitates more reliable causal inferences from Real World Evidence.
• Future work will focus on refining EA3 and exploring its broader applications in medical research and regulatory science.