関連する実験動画

Updated: Feb 4, 2026

Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment

Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment

Published on: April 19, 2024

Claude 2のバイアスリスク評価における潜在的可能性の探求：大規模言語モデルを用いたRoB 2によるランダム化比較試験の評価

Angelika Eisele-Metzger^1,2, Judith-Lisa Lieberum³, Markus Toews¹

¹Institute for Evidence in Medicine, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany.

Research synthesis methods

|February 2, 2026

Abstract:

Systematic reviews are essential for evidence-based health care, but conducting them is time- and resource-consuming. To date, efforts have been made to accelerate and (semi-)automate various steps of systematic reviews through the use of artificial intelligence (AI) and the emergence of large language models (LLMs) promises further opportunities. One crucial but complex task within systematic review conduct is assessing the risk of bias (RoB) of included studies. Therefore, the aim of this study was to test the LLM Claude 2 for RoB assessment of 100 randomized controlled trials, published in English language from 2013 onwards, using the revised Cochrane risk of bias tool ('RoB 2'; involving judgements for five specific domains and an overall judgement). We assessed the agreement of RoB judgements by Claude with human judgements published in Cochrane reviews. The observed agreement between Claude and Cochrane authors ranged from 41% for the overall judgement to 71% for domain 4 ('outcome measurement'). Cohen's κ was lowest for domain 5 ('selective reporting'; 0.10 (95% confidence interval (CI): -0.10-0.31)) and highest for domain 3 ('missing data'; 0.31 (95% CI: 0.10-0.52)), indicating slight to fair agreement. Fair agreement was found for the overall judgement (Cohen's κ: 0.22 (95% CI: 0.06-0.38)). Sensitivity analyses using alternative prompting techniques or the more recent version Claude 3 did not result in substantial changes. Currently, Claude's RoB 2 judgements cannot replace human RoB assessment. However, the potential of LLMs to support RoB assessment should be further explored.

キーワード:

GPT 人工知能自動化大規模言語モデルバイアスリスクシステマティックレビュー（主題）

さらに関連する動画

Author Spotlight: Advancements and Challenges in Surgical Treatments for Postamputation Pain

Author Spotlight: Advancements and Challenges in Surgical Treatments for Postamputation Pain

Published on: March 8, 2024

Assessment of Mouse Judgment Bias through an Olfactory Digging Task

Assessment of Mouse Judgment Bias through an Olfactory Digging Task

Published on: March 4, 2022

関連する概念動画

meta-Directing Deactivators: –NO2, –CN, –CHO, –&NoBreak;CO2R, –COR, –CO2H

meta-Directing Deactivators: –NO₂, –CN, –CHO, –⁠CO₂R, –COR, –CO₂H

All meta-directing substituents are deactivating groups. These substituents withdraw electrons from the aromatic ring, making the ring less reactive toward electrophilic substitution. For example, the nitration of nitrobenzene is 100,000 times slower than that of benzene because of the deactivating effect of the nitro group. The first step in an electrophilic aromatic substitution is the addition of an electrophile to form a resonance-stabilized carbocation. The energy diagrams for...

2° Amines to N-Nitrosamines: Reaction with NaNO2

2° Amines to N-Nitrosamines: Reaction with NaNO₂

Secondary amines react with nitrous acid to form N-nitrosamines, as depicted in Figure 1. Nitrous acid, a weak and unstable acid, is formed in situ from an aqueous solution of sodium nitrite and strong acids, such as hydrochloric acid or sulfuric acid, in cold conditions. In the presence of an acid, the nitrous acid gets protonated. The subsequent loss of water results in the formation of the electrophile known as nitrosonium ion.

Design Example: Analyzing Capacity Contours for Flood Risk Assessment

Design Example: Analyzing Capacity Contours for Flood Risk Assessment

Flood risk assessment involves careful planning and analysis to ensure the safety of communities near water retention structures. Capacity contours are a vital tool in this process, as they illustrate the potential spread of water at specific levels in a given area. In the context of building a bund across a small valley, these contours play a critical role in evaluating the safety of nearby residential areas.In this example, the bund is intended to store stormwater in the valley. The engineers...

Confirmation Biases

Confirmation Biases

The confirmation bias is the tendency to focus on information that confirms our existing beliefs and ignore information that is inconsistent with our expectations. For example, if you think that your professor is not very nice, you notice all of the instances of rude behavior exhibited by the professor while ignoring the countless pleasant interactions he is involved in on a daily basis. Have you ever fallen prey to the confirmation bias, either as the source or target of such bias?

SN2 Reaction: Kinetics

S_N2 Reaction: Kinetics

Kinetic Studies and Significance
In a chemical reaction, a relationship exists between the concentration of reactants and the rate at which the reaction proceeds. The study to measure this relationship is known as the kinetics of a chemical reaction. Kinetic studies are used to deduce the rate law of a chemical reaction, which provides information about the species involved during the transition state of the rate-determining step. Thus, kinetic studies help to derive the mechanism of a...

SN2 Reaction: Mechanism

S_N2 Reaction: Mechanism

The kinetic studies of SN2 reactions suggest an essential feature of its mechanism: it is a single-step process without intermediates. Here, both the nucleophile and the substrate participate in the rate-determining step.
The presence of the more electronegative halogen in the substrate creates a polarized carbon-halide bond. The halide pulls the electron cloud generating an electrophilic center at the carbon atom. Thus, the carbon atom carries a partial positive charge while the halide has a...