Statistical derivation of cut-off values for in vitro assays

Chemical substances and mixtures are classified based on their toxicological hazard. The regulatory validation process for in vitro methods for hazard assessment assesses their relevance by comparing them to standard in vivo test data. This requires transforming continuous read-out data into ordinal data (hazard classes). Existing strategies for developing new methods often overlook the constraints associated with small datasets, omitting the use of contemporary statistical techniques such as uncer­tainty quantification and bootstrapping. To overcome these limitations, we apply bootstrapping, estimates for the out-of-sample error, and uncertainty quantification to the validation dataset for eye irritation of Kaluzhny et al. (2011) and to a dataset of plant protection products (PPPs) pub­lished by Kolle et al. (2015), which were tested for eye irritation in vitro (OECD TG 492) and in vivo (OECD TG 405). Assessment criteria for sensitivity, specificity, and accuracy are proposed, considering uncertainty quantification and estimation of the out-of-sample error. The cut-off value for PPPs based on the available set of in vitro-in vivo data pairs can be improved by using modern cut-off approaches. For PPPs, the OECD recommended cut-off of 60% mean tissue viability based on single substances leads to lower sensitivity than the newly derived cut-off value of 67%. For liquid single substances, the OECD-recommended cut-off is confirmed. This case study demonstrates that modern statistical methods for small datasets improve the reliability of in vitro cut-off values and should therefore be used to revise and derive cut-off values for hazard classification in future.