A comprehensive meta-reanalysis of the robustness of the experience-accuracy effect in clinical judgment

Experience is one of the most commonly studied variables in clinical judgment research. In a meta-analysis of research from 1970 to 1996 of judgments made by 4,607 participants from 74 studies, Spengler, White, Ægisdóttir, Maugherman, Anderson, et al. (2009) found an experience-accuracy fixed effect of d = .121 (95% CI [.06, .18]), indicating that with more experience, counseling and other psychologists obtain only modest gains in decision-making accuracy. We sought to conduct a more rigorous assessment of the experience-accuracy effect by synthesizing 40 years of research from 1970 to 2010, assessing the same and additional moderators, including subgroup analyses of extremes of experience, and conducting a sensitivity analysis. The judgments formed by 11,584 clinicians from 113 studies resulted in a random effects d of .146 (95% CI [.08, .21]), reflecting the robustness of only a small impact of experience on decision-making accuracy. The sensitivity analysis revealed that the effect is consistent across analysis and methodological considerations. Mixed effects metaregression revealed no statistically significant relation between 40 years of time and the experience-accuracy effect. A cumulative meta-analysis indicated that the experience-accuracy effect stabilized in the literature in the year 1999, after the accumulation of 82 studies, with no appreciable change since. We assessed a broader range of experience comparing no experience to some experience and comparing nonexperts with experts, and for differences as a function of decision making based on psychological tests; however, these and most other moderators were not significant. Implications are discussed for clinical decision-making research, training, and practice.