Empirical assessment of expertise

The assessment of expertise is vital both in practical situations that call for expert judgment and in theoretical research on the psychology of experts. It can be difficult, however, to determine whether a judge is in fact performing expertly. Our goal was to develop an empirical measure of expert judgment. We argue that two necessary characteristics of expertise are discrimination of the various stimuli in the domain and consistent treatment of similar stimuli. We combine measures of these characteristics to form a ratio we call the Cochran-Weiss-Shanteau (CWS) index of expertise. The proposed index was demonstrated using two studies that distinguished experts from nonexperts based on their judgmental performance. The index provides new insights into expertise and offers a partial definition of expertise that may be useful in a variety of theoretical and applied settings. Potential applications of this research include selection, training, and evaluation of experts and of expert-machine systems.