On the interpretation of test sensitivity in the two-test two-population problem: assumptions matter

Bayesian analyses of diagnostic test accuracy often require the assumption of constant test accuracy among populations to ensure model identifiability. In a prior study (Toft, N., Jørgensen, E., Højsgaard, S., 2005. Diagnosing diagnostic tests: evaluating the assumptions underlying the estimation of sensitivity and specificity in the absence of a gold standard. Prev. Vet. Med. 68, 19-33), the sensitivity estimate from a two-test two-population model was shown to be weighted toward the population with the higher prevalence of infection. In the present study, we provided analytical formulae that give insight into the effect of assuming constant sensitivity when this assumption was false. To further investigate the effect of failure of the assumption of constant sensitivity, we also simulated several data sets under the assumption that the first test's sensitivity varied in the two populations. Bayesian conditional independence models that presumed constant sensitivities were implemented in WinBUGS and posterior estimates (mean and 95% probability intervals) were evaluated based on the known true values of the parameters. Findings from the Bayesian analyses of several scenarios indicated that the posterior mean was a good estimate of the weighted mean of the sensitivities in the two populations, when one test was perfectly specific. When neither test was perfectly specific, the Bayesian posterior mean for test 1 sensitivity was either greater than the larger of the two true sensitivities, or smaller than both, and estimates of prevalence and the second test's specificity were incorrect. The implication is that estimates of some parameters will be biased if test sensitivities are not constant across populations. Without a perfectly specific test, and if the assumption of constant sensitivity fails, the only solution we are aware of would involve incorporating prior information on at least two parameters.