Related Experiment Video
Updated: Nov 30, 2025
18:30
RNA-seq Analysis of Transcriptomes in Thrombin-treated and Control Human Pulmonary Microvascular Endothelial Cells
Published on: February 13, 2013
22.2K
The Rigor and Transparency Index Quality Metric for Assessing Biological and Medical Science Methods.
Joe Menke1, Martijn Roelandse2, Burak Ozyurt3
1Center for Research in Biological Systems, UCSD, SciCrunch Inc, La Jolla, CA 92093, USA.
Iscience
|November 16, 2020
Summary
Scientific reproducibility is improving, but many rigor criteria are still missed. An automated tool, SciScore, assesses research rigor, revealing that less than half of criteria are routinely met. A new metric, Rigor and Transparency Index (RTI), shows no correlation with Journal Impact Factor.
Area of Science:
- Scientific research methodology
- Biomedical research
- Scholarly publishing
Background:
- The scientific community faces a reproducibility crisis stemming from ingrained practices.
- Journals are implementing checklists to improve rigor and reproducibility.
- Existing methods for assessing research quality are insufficient.
Purpose of the Study:
- To develop an automated tool (SciScore) for evaluating research articles adherence to rigor criteria.
- To assess the scale of adherence to key rigor criteria in scientific publications.
- To introduce a new journal quality metric, the Rigor and Transparency Index (RTI).
Main Methods:
- Development of SciScore, an automated tool assessing research articles against rigor criteria (e.g., NIH criteria, RRIDs).
- Analysis of a large scale of research articles to quantify adherence to criteria like blinding and power analysis.
- Calculation of the Rigor and Transparency Index (RTI) for journals based on average adherence scores.
- Comparison of RTI with Journal Impact Factor (JIF).
Main Results:
- SciScore demonstrated the ability to evaluate research articles at an unprecedented scale.
- Despite improvements, less than half of key rigor criteria are routinely addressed by authors.
- The Rigor and Transparency Index (RTI) showed no correlation with the Journal Impact Factor.
- Specific checklists influenced average journal scores.
Conclusions:
- Automated tools like SciScore can efficiently assess research rigor and transparency.
- The RTI may serve as a more reliable proxy for methodological quality than the JIF.
- Further efforts are needed to improve adherence to essential research rigor criteria.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
280
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
280
01:11Variability: Analysis
330
Measures of variability are statistical metrics that reveal the dispersion pattern within a dataset. They are pivotal in biostatistics, providing insights into the heterogeneity within health and biological data. Variability signifies the degree to which data points diverge from one another, helping researchers understand the potential range of values and associated uncertainty within the data.
The range is a simple measure of variability, indicating the difference between the highest and...
The range is a simple measure of variability, indicating the difference between the highest and...
330
01:11Statistical Analysis: Overview
12.6K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
12.6K
03:50The Scientific Method
63.4K
Chemistry is an empirical science. Scientists often pose questions to understand the chemistry in everyday life and seek answers to these questions. To achieve this, scientists follow a definitive series of steps that together make up the Scientific Method. This approach involves making observations, asking questions, building a hypothesis, conducting experiments, analyzing results, and forming a conclusion.
63.4K
01:32The Scientific Method
254.3K
The scientific method is a detailed, empirical problem-solving process used by biologists and other scientists. This iterative approach involves formulating a question based on observation, developing a testable potential explanation for the observation (called a hypothesis), making and testing predictions based on the hypothesis, and using the findings to create new hypotheses and predictions.
Generally, predictions are tested using carefully-designed experiments. Based on the outcome of these...
Generally, predictions are tested using carefully-designed experiments. Based on the outcome of these...
254.3K
02:40The Scientific Method
64.3K
Research is what makes the difference between facts and opinions. Facts are observable realities, and opinions are personal judgments, conclusions, or attitudes that may or may not be accurate. In the scientific community, facts can be established only using evidence collected through empirical research.
64.3K