Statistical discrimination in the presence of selection effects

Discrimination between diseases is a complex task. Cases may present contradictory information and diseases can present with unusual or atypical symptoms. In many diagnostic problems the recorded diagnosis is either a true diagnosis, based on hard evidence, or a working diagnosis, not necessarily equivalent to the true underlying disease with an associated level of uncertainty. This problem is often confounded since the type of diagnosis given may be subjected to selection effects. Much medical data is categorical in nature, hence existing techniques for identifying selection effects are inappropriate. This paper provides a method of obtaining a single parameter modelling, the probability of giving a true diagnosis dependent on the nature of the true disease, thereby offering a simple measure for the presence of selection effects. When the size of the data is limited identifiability problems exist with calculating this parameter, however this paper shows how a sensitivity analysis based on the profile likelihood can be used to identify the presence of selection effects even in this difficult situation.