Extrapolation of cytotoxic masked effects in planar in vitro assays

The masking of specific effects in in vitro assays by cytotoxicity is a commonly known phenomenon. This may result in a partial or complete loss of effect signals. For common in vitro assays, approaches for identifying and quantifying cytotoxic masking are partly available. However, a quantification of cytotoxicity-affected signals is not possible. As an alternative, planar bioassays that combine high-performance thin layer chromatography with in vitro assays, such as the planar yeast estrogen screen (p-YES), might allow for a quantification of cytotoxically affected signals. Affected signals form a typical ring structure with a supressed or completely lacking centre that results in a double peak chromatogram. This study investigates whether these double peaks can be used for fitting a peak function to extrapolate the theoretical, unaffected signals. The precision of the modelling was evaluated for four individual peak functions, using 42 ideal, undistorted peaks from estrogenic model compounds in the p-YES. Modelled ED₅₀-values from bisphenol A (BPA) experiments with cytotoxically disturbed signals were 13 times higher than for the apparent data without compensation for cytotoxicity (320 ± 63 ng versus 24 ± 17 ng). This finding has a high relevance for the modelling of mixture effects according to concentration addition that requires unaffected, complete dose-response relationships. Finally, we applied the approach to results of a p-YES assay on leachate samples of an elastomer material used in water engineering. In summary, the fitting approach enables the quantitative evaluation of cytotoxically affected signals in planar in vitro assays and also has applications for other fields of chemical analysis like distorted chromatography signals.