A machine learning-based analysis method for small molecule high content screening of three-dimensional cancer...

Although multiparameter cellular morphological profiling methods and three-dimensional (3D) biological model systems can potentially provide complex insights for pharmaceutical discovery campaigns, there have been relatively few reports combining these experimental approaches. In this study, we used the U87 glioblastoma cell line grown in a 3D spheroid format to validate a multiparameter cellular morphological profiling screening method. The steps of this approach include 3D spheroid treatment, cell staining, fully automated digital image acquisition, image segmentation, numerical feature extraction, and multiple machine learning approaches for cellular profiling. For comparison, we measured the same samples after live-cell microscopy with an endpoint CellTiter-Glo cell viability assay. The combined method characterized 7 reference compounds with previously reported anticancer/cytotoxic properties that induce quantifiably different spheroid morphologies in this assay. The method was then used to screen a library of 925 compounds that are related to kinase signaling pathways. Both unsupervised and supervised machine learning approaches identified compounds that induced morphologies similar to those induced by the reference compounds. We performed a follow-up 16-point concentration response experiment for 3 of these compounds selected from our profiling pipeline and confirmed their phenotype. The morphology-based concentration response for these compounds was also correlated with the CellTiter-Glo endpoint assay. Additionally, the measured morphological phenotypes displayed different enrichment levels of commonly annotated mechanisms of action. Our analysis was able to identify selected mechanisms of action associated with specific phenotypic signatures. Overall, the presented screening and analysis method can distinguish between different spheroid structural changes that are caused by specific candidate anticancer compounds. SIGNIFICANCE STATEMENT: Morphological profiling has become a powerful tool in the field of microscopy for finding distinct mechanisms of action groups and small molecule screening to identify new phenotypes. This study presents new potential mechanisms of action groups for known glioblastoma candidates from the screening library, which is believed to help advance the search for more effective glioblastoma therapies.