A new modeling approach for quantifying expert opinion in the drug discovery process

Expert opinion plays an important role when choosing clusters of chemical compounds for further investigation. Often, the process by which the clusters are assigned to the experts for evaluation, the so-called selection process, and the qualitative ratings given by the experts to the clusters (chosen/not chosen) need to be jointly modeled to avoid bias. This approach is referred to as the joint modeling approach. However, misspecifying the selection model may impact the estimation and inferences on parameters in the rating model, which are of most scientific interest. We propose to incorporate the selection process into the analysis by adding a new set of random effects to the rating model and, in this way, avoid the need to model it parametrically. This approach is referred to as the combined model approach. Through simulations, the performance of the combined and joint models was compared in terms of bias and confidence interval coverage. The estimates from the combined model were nearly unbiased, and the derived confidence intervals had coverage probability around 95% in all scenarios considered. In contrast, the estimates from the joint model were severely biased under some form of misspecification of the selection model, and fitting the model was often numerically challenging. The results show that the combined model may offer a safer alternative on which to base inferences when there are doubts about the validity of the selection model. Importantly, thanks to its greater numerical stability, the combined model may outperform the joint model even when the latter is correctly specified.