Diagnostic illusions: the reliability of random chance

A "Monte Carlo" experiment was performed in order to determine chance concordance rates for multiple test scenarios often encountered in chiropractic diagnosis. The Monte Carlo simulation took into account the following variables: the number of tests involved in the diagnosis; the number of vertebral segments implicated by each test; the proportion of tests in agreement relative to the number of tests performed; and the segmental margin of error accepted. Random data for up to five diagnostic tests performed on 500 "patients" were computer generated and a wide variety of test scenarios analyzed. One typical analysis asked: if four diagnostic tests are performed on each patient, each test implicating on average three vertebral segments, and a plus or minus one segment error margin is accepted, what are the chance odds that any three out of the four tests will implicate the same vertebral segment? The answer, determined by simply counting how often this happened in the 500 "patients," was 89%. Many test scenarios yielded chance levels much higher than might have been expected. High probability situations as well as those test criteria yielding relatively low chance concordance rates are identified.