Continuously variable rating: a new, simple and logical procedure to evaluate original scientific publications

Area of Science:

• Bibliometrics
• Scholarly Communication
• Information Science

Background:

• Impact Factors (IF) are commonly used to assess individual research articles.
• However, the skewed distribution of citations limits the reliability of IF for article-level evaluation.
• This study addresses the asymmetry inherent in citation data.

Purpose of the Study:

• To quantify citation distribution asymmetry in academic journals.
• To propose and validate a novel computer-based procedure for evaluating individual articles.
• To offer an alternative to traditional Impact Factor limitations.

Main Methods:

• Analysis of citation symmetry across journal clusters based on Impact Factor.
• Calculation of normalized cumulative citations and decile rankings.
• Estimation of Positive Predictive Value (PPV) for different IF-defined evaluation classes (LOW, MID, HIGH).
• Development of a Continuously Variable Rating (CVR) procedure for article evaluation.

Main Results:

• Citation distributions exhibited skewness consistent with the Pareto Distribution (1
• Observed Positive Predictive Values ranged from 24% to 43% for most journals in the ISI database.
• The proposed Continuously Variable Rating method demonstrated effectiveness.

Conclusions:

• Citation data displays significant skewness, impacting traditional evaluation metrics.
• Continuously Variable Rating offers a valid and accurate method for rating individual articles.
• This computer-based approach provides a more nuanced evaluation within a specific journal and publication year.