Related Experiment Video
Updated: Jun 14, 2025
Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment
Published on: April 19, 2024
A survey of experts to identify methods to detect problematic studies: stage 1 of the INveStigating ProblEmatic
Jack Wilkinson1, Calvin Heal1, George A Antoniou2
1Centre for Biostatistics, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK.
Background And Objective:
Randomized controlled trials (RCTs) inform health-care decisions. Unfortunately, some published RCTs contain false data, and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesize all RCTs which have been conducted on a given topic. This means that any of these 'problematic studies' are likely to be included, but there are no agreed methods for identifying them. The INveStigating ProblEmatic Clinical Trials in Systematic Reviews (INSPECT-SR) project is developing a tool to identify problematic RCTs in systematic reviews of health care-related interventions. The tool will guide the user through a series of 'checks' to determine a study's authenticity. The first objective in the development process is to assemble a comprehensive list of checks to consider for inclusion.
Methods:
We assembled an initial list of checks for assessing the authenticity of research studies, with no restriction to RCTs, and categorized these into five domains: Inspecting results in the paper; Inspecting the research team; Inspecting conduct, governance, and transparency; Inspecting text and publication details; Inspecting the individual participant data. We implemented this list as an online survey, and invited people with expertise and experience of assessing potentially problematic studies to participate through professional networks and online forums. Participants were invited to provide feedback on the checks on the list, and were asked to describe any additional checks they knew of, which were not featured in the list.
Results:
Extensive feedback on an initial list of 102 checks was provided by 71 participants based in 16 countries across five continents. Fourteen new checks were proposed across the five domains, and suggestions were made to reword checks on the initial list. An updated list of checks was constructed, comprising 116 checks. Many participants expressed a lack of familiarity with statistical checks, and emphasized the importance of feasibility of the tool.
Conclusion:
A comprehensive list of trustworthiness checks has been produced. The checks will be evaluated to determine which should be included in the INSPECT-SR tool.
Insights
A new tool is being developed to identify problematic randomized controlled trials (RCTs) in systematic reviews. This involves creating a comprehensive list of checks to ensure the authenticity of research studies.
Area of Science:
- Clinical Trials Methodology
- Health Research Integrity
- Systematic Review Practices
Background:
- Randomized controlled trials (RCTs) are crucial for healthcare decisions, but some contain fabricated or false data.
- Systematic reviews aim to synthesize all relevant RCTs, potentially including problematic studies.
- There is a lack of standardized methods for identifying unreliable RCTs within systematic reviews.
Purpose of the Study:
- To develop the INveStigating ProblEmatic Clinical Trials in Systematic Reviews (INSPECT-SR) tool.
- To create a comprehensive list of checks for assessing the authenticity of clinical trials.
- To guide users in evaluating the trustworthiness of research studies.
Main Methods:
- An initial list of 102 checks was developed across five domains: results, research team, conduct/governance/transparency, text/publication details, and participant data.
- An online survey was used to gather feedback from 71 international experts on the initial list.
- Participants provided feedback, suggested new checks, and proposed revisions to existing ones.
Main Results:
- The feedback resulted in an updated list of 116 checks.
- Fourteen new checks were added, and existing checks were reworded.
- Experts emphasized the importance of feasibility and expressed unfamiliarity with some statistical checks.
Conclusions:
- A comprehensive list of trustworthiness checks for RCTs has been established.
- These checks will be further evaluated for inclusion in the INSPECT-SR tool.
- The INSPECT-SR tool aims to improve the reliability of systematic reviews by identifying problematic studies.
More Related Videos
06:55Inverse Probability of Treatment Weighting Propensity Score using the Military Health System Data Repository and National Death Index
Published on: January 8, 2020
06:26Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images SDM-PSI
Published on: November 27, 2019
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
02:07Blind Procedures
01:12Hazard Ratio
For example, in a clinical trial...
01:20Study Designs in Epidemiology
Observational studies are those where the researcher does not intervene but rather observes natural variations. They include cross-sectional, cohort, and...
01:11Blinding
01:25Strategies for Assessing and Addressing Confounding
Confounding can be addressed at both the design phase of a study and through analytical methods after data...