Assessing physicians' accuracy in diagnosing paediatric patients with acute abdominal pain: measuring accuracy for...

In evaluating the accuracy of diagnostic tests there are many situations where the true disease status can be one of multiple disorders. For example, when paediatric patients suffer acute abdominal pain, the underlying diagnosis could be appendicitis, intestinal obstruction, gastroenteritis, urinary tract infection, etc. In this paper we describe a format for collecting data for the case of multiple truth states. This new format allows one to make pairwise comparisons of the diagnostic accuracy between all of the different truth states. We propose a summary measure of accuracy which is a weighted average of the pairwise estimates of accuracy. Estimators are derived for the variance and covariance of the estimated summary accuracy. The small sample properties of the estimators are evaluated in a Monte Carlo simulation study. The new data collection format and summary measure were used in a paediatric acute abdominal pain study; data from this study are used to illustrate the methods.