自动提取大语言模型随机对照试验的数值结果
Hye Sun Yun1, David Pogrebitskiy1, Iain J Marshall2
1Northeastern University, Boston, MA, USA.
Proceedings of machine learning research
|September 22, 2025
概括
大型语言模型 (LLM) 通过从随机对照试验 (RCT) 提取数据来实现自动化元分析的承诺. 虽然对简单的结果有效,但LLM在需要推断的复杂数据上扎.
科学领域:
- 医疗信息学 医疗信息学
- 自然语言处理自然语言处理.
- 临床试验 临床试验
背景情况:
- 分析对于可靠的治疗有效性估计至关重要,它综合了多项随机对照试验 (RCT) 的研究结果.
- 目前的元分析需要从单个试验报告中进行繁忙的手动数据提取,从而限制了效率和可扩展性.
- 使用语言技术实现这种数据提取的自动化可以实现按需的元分析.
研究的目的:
- 评估现代大型语言模型 (LLM) 在可靠地从临床试验报告中提取数值发现以进行元分析的能力.
- 评估与干预,比较器和结果相关的数值结果的零射击条件提取中的LLM性能.
主要方法:
- 开发和发布一个细粒度评估数据集的临床试验报告与注释数值发现.
- 在注释数据集上使用零射击方法评估七个大型语言模型 (LLM).
- 专注于从试验报告中提取干预,比较器和结果的数值结果.
主要成果:
- 大规模的LLM显示出完全自动元分析的近乎能力,特别是对于像死亡率这样的二分化结果.
- 当结果测量是复杂的并且需要推断推理来计算结果时,LLM表现不佳.
- 即使在生物医学文本培训的LLM中,性能限制仍然存在.
结论:
- 大型语言模型 (LLM) 正在接近随机对照试验 (RCT) 的全自动元分析的目标.
- 目前的LLM在从试验报告中提取和合成复杂的数值数据方面面临重大限制.
- 需要在LLM方面取得进一步的进展,以克服用于综合元分析的推理数据处理方面的挑战.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
03:37Author Spotlight: Impact of Intergenic Interactions on Disease-Identifying Dark Biomarkers
Published on: March 1, 2024
1.2K
相关概念视频
01:13Randomized Experiments
8.9K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
8.9K
02:07Improving Translational Accuracy
14.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.1K
02:07Improving Translational Accuracy
3.6K
3.6K
01:52Regression Toward the Mean
6.9K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.9K
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
407
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
407
01:12Statistical Software for Data Analysis and Clinical Trials
1.4K
Statistical software is pivotal in data analysis and clinical trials by providing tools to analyze data, draw conclusions, and make predictions. These software packages range from simple data management applications to complex analytical platforms, supporting various statistical tests, models, and simulation techniques. Their significance lies in their ability to handle vast amounts of data with precision and efficiency, enabling researchers to validate hypotheses, identify trends, and make...
1.4K