Related Experiment Videos
Artificial Intelligence-enabled GRADE: how the GRADE Working Group will use automation to rate the certainty of
Bernardo Sousa-Pinto1, Manuel Marques-Cruz1, Rafael José Vieira1
1Faculty of Medicine, MEDCIDS - Department of Community Medicine, Information and Health Decision Sciences, University of Porto, Porto, Portugal; CINTESIS@RISE - Health Research Network, Faculty of Medicine, MEDCIDS, University of Porto, Porto, Portugal.
Objectives:
The Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group is developing GRADErater (GRADE Rating Automation Through Enhanced Reasoning), an official, automated tool for evaluation of the certainty of evidence (CoE). In this article, we describe the principles and methods underlying its development and state how the GRADE Working Group (GWG) will use automation to rate the CoE of intervention effects.
Methods:
We followed the GRADE methods for registering a project group on GRADE and artificial intelligence (AI). The project group established that the automated tool should be developed according to the following principles: (i) compliance with the current GRADE guidance, (ii) transparency, (iii) human oversight, (iv) implementation of decision rules, (v) ease of use, (vi) understandability, and (vii) continuous improvement. We developed a set of decision rules to appraise each domain of the CoE in pairwise and network meta-analysis. These rules were developed based on official GRADE sources and validated by GRADE experts. Based on these rules, we created a first version (https://gradeai.med.up.pt/) for GRADErater. We are now evaluating this version in terms of (i) its underlying rules and (ii) its user interface. These assessments will allow for a refinement of the rules and of the first version. The modified version will be appraised and presented to the GWG with input from internal and external interest-holders before we seek formal approval. Once launched, the automated tool will be continuously evaluated and refined by incorporating feedback from end users. We will add new features (including generative AI-based functionalities), integrate this tool with GRADEpro, and develop versions in languages other than English.
Conclusion:
This project will follow a transparent methodology to create GRADErater, an official tool endorsed by the GWG that will support humans in applying the most current GRADE methods to rate the CoE.
Related Concept Videos
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Randomized Experiments
Simple randomization
Simple...
Statistical Significance
Group Design