Interexaminer reliability for multiple diagnostic test regimens

A methodology is presented for conducting interexaminer reliability studies when indication for adjustment is based on the outcomes of multiple diagnostic procedures. The adjustment decision-making process is simulated by defining multitest scores to be computed from test battery results and defining threshold values of these scores to demarcate the selection/rejection decision made by the clinician. A probability analysis of random chance agreement suggests that intermediate values of the threshold multitest scores must be used to ensure moderate agreement between observers and the viability of concordance statistics.