High-impact and transformative science (HITS) metrics: Definition, exemplification, and comparison

Area of Science:

• Bibliometrics and Scientometrics
• Research Evaluation
• Scientific Impact Assessment

Background:

• Identifying high-impact and transformative scientific research is crucial for funding and policy.
• Existing metrics may not fully capture the nuances of research impact and transformation.

Purpose of the Study:

• To present and validate a novel set of text- and citation-based metrics for identifying high-impact and transformative research.
• To analyze the relationship between transformativeness and impact, and their underlying components.

Main Methods:

• Developed 11 novel metrics across seven categories (e.g., Radical-Generative, Risky, Multidisciplinary).
• Applied metrics to 10,778,696 MEDLINE articles matched with Science Citation Index Expanded data (1983-2012).
• Utilized field-period pair analysis and factor analysis for validation and comparison.

Main Results:

• A positive correlation (ρ = .402) exists between transformativeness and impact, though they are distinct.
• No evidence suggests transformative work is adopted slowly or that new concepts obsolete old ones.
• Generation of new concepts and highly cited work is associated with higher risk.
• Research drawing from diverse fields demonstrates wider usage.

Conclusions:

• The novel metrics effectively differentiate and quantify aspects of research impact and transformativeness.
• Understanding the risk and multidisciplinary nature of transformative research can inform strategic promotion.
• Interdisciplinary research is linked to broader dissemination and impact.