Related Experiment Video
Updated: Nov 24, 2025
Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
A rule-based approach for automatically extracting data from systematic reviews and their updates to model the risk
Rabia Bashir1, Adam G Dunn1,2, Didi Surian1
1Centre for Health Informatics, Australian Institute of Health Innovation, Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney, New South Wales, Australia.
Abstract:
Few data-driven approaches are available to estimate the risk of conclusion change in systematic review updates. We developed a rule-based approach to automatically extract information from reviews and updates to be used as features for modelling conclusion change risk. Rules were developed to extract relevant information from published Cochrane reviews and used to construct four features: the number of included trials and participants in the reviews, a measure based on the number of participants, and the time elapsed between the search dates. We compared the performance of random forest, decision tree, and logistic regression to predict the conclusion change risk. The performance was measured by accuracy, precision, recall, F1 -score, and area under ROC (AU-ROC). One rule was developed to extract the conclusion change information (96% accuracy, 100 reviews), one for the search date (100% accuracy, 100 reviews), one for the number of included clinical trials (100% accuracy, 100 reviews), and 22 for the number of participants (97.3% accuracy, 200 reviews). For unseen reviews, the random forest classifier showed the highest accuracy (80.8%) and AU-ROC (0.80). All classifiers showed relatively similar performance with overlapping 95% confidence interval (CI). The coverage score was shown to be the most useful feature for predicting the conclusion change risk. Features mined from Cochrane reviews and updates can estimate conclusion change risk. If data from more published reviews and updates were made accessible, data-driven methods to predict the conclusion change risk may be a feasible way to support decisions about updating reviews.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:23Mechanistic Models: Compartment Models in Individual and Population Analysis
01:14Analysis Methods of Pharmacokinetic Data: Model and Model-Independent Approaches
The model approach uses mathematical models to describe changes in drug concentration over time. Pharmacokinetic models help characterize drug behavior in patients, predict drug concentration in the body fluids, calculate optimum dosage regimens, and evaluate the risk of toxicity. However, ensuring that the model fits the experimental data accurately...
01:25Strategies for Assessing and Addressing Confounding
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
01:19Pharmacovigilance
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
01:12Hazard Ratio
For example, in a clinical trial...