MetaMind: A multi-agent transformer-driven framework for automated network meta-analyses.

Area of Science:

• Biomedical Informatics
• Computational Biology
• Evidence Synthesis

Background:

• Network meta-analysis (NMA) combines direct and indirect evidence to compare multiple interventions.
• Traditional NMA processes are time-consuming, often taking months due to manual data identification, extraction, and modeling.
• Delays in NMA updates can hinder timely therapeutic area advancements.

Purpose of the Study:

• To develop and validate MetaMind, an automated, end-to-end framework for network meta-analysis.
• MetaMind aims to minimize human input in NMA processes, from study retrieval to analysis execution.
• The framework leverages transformer technology for efficient and accurate evidence synthesis.

Main Methods:

• MetaMind integrates Promptriever for semantic retrieval of clinical trials from PubMed.
• A multi-agent LLM architecture (Mixture of Agents) extracts PICO (Population, Intervention, Comparison, Outcome) data.
• GPT-4o generates scripts for Bayesian random-effects NMA and other NMA designs, validated against manual analyses in ulcerative colitis and Crohn's disease.

Main Results:

• Promptriever demonstrated superior similarity scores and relevance ranges compared to baseline models.
• Promptriever achieved high performance metrics: 82.1% recall, 91.1% precision, and 86.4% F1 score.
• MetaMind exhibited 100% accuracy in PICO extraction for remission endpoints and produced NMA results closely matching manual analyses.

Conclusions:

• MetaMind drastically reduces the NMA process duration to under a week, compared to several months for manual workflows.
• The framework maintains statistical rigor, ensuring the reliability of the synthesized evidence.
• MetaMind shows potential for scaling evidence synthesis across diverse therapeutic areas.