Linkage detection under heterogeneity and the mixture problem

Linkage analysis has contributed to the localization of many human disease genes. The presence of locus heterogeneity reduces statistical power and can prejudice the detection of linkage if the analysis assumes homogeneity. Nevertheless, mixed genetic models are not routinely used in gene searches. The null distribution of the test statistic is not uniquely defined. In this paper, a transformation is used to determine an approximate asymptotic distribution of the test statistic under a mixture model. The equivalent critical values of the test are computed and the performance of the test under various levels of heterogeneity and family size is investigated. For gene searches, we recommend the routine use of an admixture model with a critical lod score of 3.44.