Citation gaming induced by bibliometric evaluation: A country-level comparative analysis

Area of Science:

• Bibliometrics and Scientometrics
• Science Policy and Research Evaluation

Background:

• Global research evaluation systems increasingly use quantitative indicators to assess researcher performance.
• The behavioral effects of these evaluation systems on researchers remain largely unexamined.

Purpose of the Study:

• To investigate the impact of national research evaluation systems on researcher behavior.
• To introduce and analyze a novel 'inwardness' indicator to measure scientific self-referentiality.

Main Methods:

• Defined 'inwardness' as the proportion of a country's citations originating domestically versus total citations received.
• Conducted a comparative trend analysis of inwardness for G10 countries (2000-2016).
• Examined the correlation between inwardness trends and the implementation of national bibliometric evaluation policies.

Main Results:

• Observed a significant increase in Italian inwardness post-2011, coinciding with new bibliometric regulations.
• Italy exhibited the highest global inwardness and lowest international collaboration rates among G10 nations.
• This trend suggests strategic citation practices, including self-citation and citation clubs, within the Italian scientific community.

Conclusions:

• National evaluation systems can significantly shape scientific community behavior and citation practices.
• The Italian case provides critical insights into the constitutive effects of evaluation metrics in science policy.
• This study offers a potential paradigm for understanding indicator use in science-policy debates.