Related Experiment Video
Updated: May 17, 2026
07:50
Global and Current Research Trends of Single-Cell Sequencing in Cancer: A Bibliometric and Visualization Study
Published on: April 18, 2025
Counting highly cited papers for university research assessment: conceptual and technical issues
1Centro de Biotecnología y Genómica de Plantas, Universidad Politécnica de Madrid, Madrid, Spain. alonso.rodriguez@upm.es
Plos One
|October 17, 2012
Summary
Research assessment should focus on high-impact papers, not total output. Analyzing the top 1% of citations better reveals scientific progress and justifies research investments, unlike traditional metrics.
Area of Science:
- Bibliometrics and scientometrics
- Research evaluation methodologies
- Science policy and innovation studies
Background:
- Traditional research performance indicators, such as total publications and citations, may not accurately reflect scientific progress.
- Kuhnian paradigms suggest scientific progress relies on infrequent, significant breakthroughs rather than continuous 'normal science'.
- Elite research universities are characterized by substantial research investment and a high concentration of top scientists, not just publication volume.
Purpose of the Study:
- To propose and validate bibliometric indicators that accurately measure a university's contribution to scientific progress.
- To investigate the utility of the 1% high-citation tail as a robust metric for research impact.
- To differentiate genuine scientific breakthroughs from less impactful publications that may inflate traditional metrics.
Main Methods:
- Analysis of citation distributions, focusing on the characteristics of the 1% high-citation tail.
- Power-law analysis to model the frequency of highly cited papers and predict breakthroughs.
- Comparative evaluation of the 1% high-citation tail metric against traditional indicators like total papers, citations, and the h-index.
- Quantification of the impact of multinational and other non-breakthrough papers on citation rankings.
Main Results:
- Indicators within the 1% high-citation tail effectively capture contributions to scientific progress and justify research investments.
- The frequency of papers in the upper tail of the citation distribution can predict the occurrence of highly cited breakthroughs.
- Multinational papers and other less impactful publications create a false impression of excellence, particularly in lower-ranked institutions.
- The h-index obscures significant differences between higher- and lower-ranked universities due to the non-proportional representation of top-tier papers in the h-core.
Conclusions:
- Focusing on the 1% high-citation tail offers a more accurate assessment of research impact and scientific progress than traditional metrics.
- This approach provides a quantifiable basis for understanding and justifying substantial investments in elite research universities.
- Bibliometric analysis should prioritize identifying genuine scientific breakthroughs to improve the reliability of research assessment.
Related Concept Videos
01:15Study Design in Statistics
A study design is a set of techniques that allow a researcher to collect and analyze data from different variables defined for a specific research problem. Statistics is commonly for effective study design and more robust experiments,
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
Does aspirin reduce the risk of heart attacks? Is one brand of fertilizer more effective at growing roses than another? Is fatigue as dangerous to a driver as the influence of alcohol? Questions like these are answered using randomized experiments with proper...
01:11Statistical Analysis: Overview
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
01:20Biostatistics: Overview
Biostatistics plays a crucial role in understanding and analyzing data in healthcare and biology. Biostatisticians conduct experiments, gather evidence, and draw meaningful conclusions using statistical methods and techniques. Different variables form the foundation of biostatistical analysis, allowing researchers to understand and interpret data effectively. These variables are classified into different types, each serving a specific purpose in statistical analysis.
Discrete variables are...
Discrete variables are...
01:15Systematic Error: Methodological and Sampling Errors
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
02:39Indicators
Certain organic substances change color in dilute solution when the hydronium ion concentration reaches a particular value. For example, phenolphthalein is a colorless substance in any aqueous solution with a hydronium ion concentration greater than 5.0 × 10−9 M (pH < 8.3). In more basic solutions where the hydronium ion concentration is less than 5.0 × 10−9 M (pH > 8.3), it is red or pink. Substances such as phenolphthalein, which can be used to determine the pH of a solution, are called...
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...