Related Experiment Video
Updated: Sep 6, 2025
05:02
Comparing Bibliometric Analysis Using PubMed, Scopus, and Web of Science Databases
Published on: October 24, 2019
32.0K
Establishing Institutional Scores With the Rigor and Transparency Index: Large-scale Analysis of Scientific Reporting
Joe Menke1,2, Peter Eckmann2,3, Ibrahim Burak Ozyurt2,3
1Center for Research in Biological Systems, University of California, San Diego, La Jolla, CA, United States.
Journal of Medical Internet Research
|June 27, 2022
Summary
The Rigor and Transparency Index (RTI) automatically assesses scientific reproducibility, showing improvements over time. However, current average scores indicate more work is needed for full replication.
Area of Science:
- Bibliometrics
- Scientific Reproducibility
- Computational Linguistics
Background:
- Manual assessment of scientific transparency is challenging.
- Improving rigor and transparency is crucial for scientific reproducibility.
Purpose of the Study:
- To enhance the Rigor and Transparency Index (RTI) for automated assessment.
- To evaluate the rigor and transparency of journals, institutions, and countries.
Main Methods:
- Utilized natural language processing (NLP) and regular expressions to analyze over 2 million papers.
- Developed RTI version 2.0 to track 27 entity types based on reproducibility guidelines.
Main Results:
- Rigor and transparency measures generally improved from 1997-2020 (RTI 2.29 to 4.13).
- Top-performing journals, institutions (e.g., Capital Medical University), and countries (e.g., Ethiopia, Norway) were identified.
- Replication studies had significantly higher RTI scores (7.61) than original papers (3.39), validating RTI as a proxy for replicability.
Conclusions:
- The enhanced RTI serves as a reliable proxy for scientific replicability.
- Current average RTI scores for journals, institutions, and countries lag behind those of replication studies.
- Further improvements in manuscript reporting are necessary to achieve target replication standards.
Related Concept Videos
00:54Ratio Level of Measurement
19.1K
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated....
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated....
19.1K
01:11Statistical Analysis: Overview
7.3K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
7.3K
01:09Empirical Method to Interpret Standard Deviation
5.4K
The empirical rule, also known as the three-sigma rule, allows a statistician to interpret the standard deviation in a normally distributed dataset. The rule states that 68% of the data lies within one standard deviation from the mean, 95% lies within two standard deviations from the mean, and 99.7% lies within three standard deviations from the mean. Additionally, this rule is also called the 68-95-99.7 rule.
This rule is used widely in statistics to calculate the proportion of data values...
This rule is used widely in statistics to calculate the proportion of data values...
5.4K
00:55Ordinal Level of Measurement
25.6K
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks...
Data measured using an ordinal scale are similar to nominal scale data, but there is one major difference. The ordinal scale data can be ordered. An example of ordinal scale data is a list of the top five national parks...
25.6K
01:10Review and Preview
7.7K
In statistics, several tools are used to interpret the data. Measures of central tendency represent the characteristics of the data, such as mean, median, and mode. Additionally, measures of variance like standard deviation and range are used to find the spread of data from the mean. Relative standing measures the distance between data locations. Commonly used measures of relative standings are percentile, z score, and quartiles.
Percentiles are a type of fractile that partition data into...
Percentiles are a type of fractile that partition data into...
7.7K
01:04Significance Testing: Overview
3.7K
Significance testing is a set of statistical methods used to test whether a claim about a parameter is valid. In analytical chemistry, significance testing is used primarily to determine whether the difference between two values comes from determinate or random errors. The effect of a particular change in the measurement protocol, analyst, or sample itself can cause a deviation from the expected result. In the case of a suspected deviation/outlier, we need to be able to confirm mathematically...
3.7K