On the predictability of future impact in science

Area of Science:

• Bibliometrics
• Scientometrics
• Research Evaluation

Background:

• Assessing scientist research impact is crucial for recruitment and evaluations.
• Current methods primarily focus on past performance, not future potential.
• Linear regression models have been proposed for predicting future scientific impact.

Purpose of the Study:

• To critically evaluate the accuracy of linear regression models for predicting scientist future impact.
• To identify limitations and potential biases in current predictive models.
• To assess the suitability of these models for academic recruitment and evaluation.

Main Methods:

• Analysis of 762 scientist careers across physics, biology, and mathematics.
• Application of linear regression models to quantify research impact.
• Examination of model performance across different career stages.

Main Results:

• Cumulative, non-decreasing measures like the h-index exhibit intrinsic autocorrelation.
• This autocorrelation leads to a significant overestimation of predictive power.
• Model accuracy is heavily dependent on researcher career age, with poor performance for early-career scientists.

Conclusions:

• Current linear regression models for predicting scientist impact have critical flaws.
• The overestimation of predictive power and age-dependency raise doubts about their reliability.
• Further research is needed before these models can be responsibly used in recruitment decisions.