[Evaluation of scientific quality using citation analysis and other bibliometric methods]

Area of Science:

• Bibliometrics and Scientometrics
• Scientific Evaluation Methodologies

Context:

• Citation frequency, as indexed by the Science Citation Index, is widely used in scientific evaluation.
• The journal impact factor is often proposed as a quick measure of article quality.

Purpose:

• To critically examine the validity of bibliometric methods, specifically citation frequency and journal impact factor, for assessing scientific quality.
• To determine the suitability of these metrics for evaluating individual scientists, research groups, and institutions.

Summary:

• Citation frequency is influenced by citation biases and field-specific dynamics, making it an equivocal measure of scientific quality.
• Individual citation frequencies are highly variable, rendering them unsuitable for grading scientists or research groups.
• Journal impact factors are poor indicators of individual article or author quality due to skewed citation distributions and weak correlations with author citation rates.

Impact:

• Bibliometric indicators can provide relevant information on research activity at aggregate levels (e.g., institutes) when adjusted for research field effects.
• The findings question the reliability of current bibliometric practices for precise scientific quality assessment, advocating for nuanced evaluation approaches.
• This analysis highlights the limitations of using citation counts and journal impact factors as standalone metrics for scientific merit.