Comparison of visual assessment and quantitative goodness-of-fit metrics on GUTS model fits

For the application of toxicokinetic-toxicodynamic (TKTD) models in the European environmental risk assessment (ERA) of plant protection products, it is recommended to evaluate model predictions of the calibration as well as the independent validation data set based on qualitative criteria (visual assessment) and quantitative goodness-of-fit (GoF) metrics. The aims of this study were to identify whether quantitative criteria coincide with human visual perception of model performance and which evaluator characteristics influence their perception. In an anonymous online survey, > 70 calibration and validation general unified threshold models of survival (GUTS) fits were ranked by 64 volunteers with a professional interest in ecotoxicology and TKTD modeling. Participants were asked to score model fits to the time resolved survival data from toxicity experiments and to an aggregated dose-response curve representation. Dose-response curve plots tended to be scored better than time series, although both representations were based on the same toxicity test data and model results. For the time series, quantitative indices and visual assessments generally agreed on model performance. However, rankings varied with individual perceptions of the participants. Visual assessment scores were best predicted using a combination of GoF metrics. From the survey participants' majority agreement on fit acceptance, GoF cut-off criteria could be derived that indicated sufficient fit performance. The most conservative GoF criterion well resembled current suggestions by the European Food Safety Authority. Hence, the survey results provide evidence that current quantitative GUTS assessment practice in ERA is consistent with perceptions of fit quality based on visual judgements of the dynamic model behavior by a large number of practitioners. Thus, our study fosters trust in model performance assessment.