Related Experiment Video
Updated: Jul 11, 2025
14:02
Measuring Engagement of Spectators of Social Digital Games
Published on: July 3, 2021
3.5K
Quantitative research assessment: using metrics against gamed metrics
John P A Ioannidis1, Zacharias Maniadis2,3
1Departments of Medicine, of Epidemiology and Population Health, of Biomedical Data Science, and of Statistics, and Meta-Research Innovation Center at Stanford (METRICS), Stanford University, SPRC, MSOB X306, 1265 Welch Rd, Stanford, CA, 94305, USA. jioannid@stanford.edu.
Internal and Emergency Medicine
|November 3, 2023
Summary
Quantitative bibliometric indicators are misused for research assessments. This review details gaming practices like authorship and citation manipulation, suggesting better metrics can detect and deter such issues.
Area of Science:
- Bibliometrics and scientometrics
- Research assessment
- Scientific publishing
Background:
- Quantitative bibliometric indicators are extensively used in research assessment and career advancement.
- The "publish or perish" and "get cited or perish" culture incentivizes gaming of these metrics.
- Gaming practices can undermine the integrity of scientific evaluations.
Purpose of the Study:
- To review and categorize various gaming practices associated with quantitative bibliometric indicators.
- To discuss the implications of these gaming tactics on scientific careers and evaluations.
- To propose the use of improved quantitative metrics for detecting and contextualizing gaming.
Main Methods:
- Comprehensive literature review of gaming practices in bibliometrics.
- Categorization of gaming tactics into authorship-based, citation-based, editorial-based, and journal-based.
- Discussion of specific examples such as gift authorship, salami slicing, citation farms, and paper mills.
Main Results:
- Identified numerous gaming practices, including authorship manipulation (gift authorship, salami slicing), citation manipulation (self-citations, citation farms), and journal-based manipulation (impact factor gaming).
- Highlighted the role of paper mills and spurious publications in inflating metrics.
- Emphasized that quantitative metrics and analyses can aid in detecting and contextualizing these gaming behaviors.
Conclusions:
- Quantitative bibliometric indicators are susceptible to widespread gaming, impacting research assessments and careers.
- Detection and perspective on gaming practices can be enhanced through rigorous, transparent quantitative metrics.
- Incorporating indicators of both best and poor research practices into metrics can strengthen scientific evaluations.
Related Concept Videos
01:12Quantitative Analysis
309
Quantitative analysis is a technique for measuring the amount of specific constituents in a sample. When the sample's composition is unknown, qualitative analysis is performed first to identify its components, which ensures that the correct substances are measured during the quantitative phase.
In quantitative analysis, two key measurements are made: the sample quantity and a property proportional to the amount of the analyte (the substance being analyzed). This forms the basis of the...
In quantitative analysis, two key measurements are made: the sample quantity and a property proportional to the amount of the analyte (the substance being analyzed). This forms the basis of the...
309
01:02Quantifying and Rejecting Outliers: The Grubbs Test
1.6K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.6K
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
133
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
133
01:11Statistical Analysis: Overview
6.6K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
6.6K
01:10Statgraphics
134
Statgraphics is a comprehensive statistical software suite designed for both basic and advanced data analysis. Originating in 1980 at Princeton University under Dr. Neil W. Polhemus, it was one of the pioneering tools for statistical computing on personal computers, with its public release in 1982 marking an early milestone in data science software. Over the years, it has evolved into a robust platform for data science, offering tools for regression analysis, ANOVA, multivariate statistics,...
134
00:54Ratio Level of Measurement
18.2K
The way a set of data is measured is called its level of measurement. Correct statistical procedures depend on a researcher being familiar with levels of measurement. For analysis, data are classified into four levels of measurement—nominal, ordinal, interval, and ratio.
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated....
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated....
18.2K