Automated cell annotation and classification on histopathology for spatial biomarker discovery

Histopathology with hematoxylin and eosin (H&E) staining is routinely employed for clinical diagnoses. Single-cell analysis of histopathology provides a powerful tool for understanding the intricate cellular interactions underlying disease progression and therapeutic response. However, existing efforts are hampered by inefficient and error-prone human annotations. Here, we present an experimental and computational approach for automated cell annotation and classification on H&E-stained images. Instead of human annotations, we use multiplexed immunofluorescence (mIF) to define cell types based on cell lineage protein markers. By co-registering H&E images with mIF of the same tissue section at the single-cell level, we create a dataset of 1,127,252 cells with high-quality annotations on tissue microarray cores. A deep learning model combining self-supervised learning with domain adaptation is trained to classify four cell types on H&E images with an overall accuracy of 86%-89%, and the cell classification model is applicable to whole slide images. Further, we show that spatial interactions among specific immune cells in the tumor microenvironment are linked to patient survival and response to immune checkpoint inhibitors. Our work provides a scalable approach for single-cell analysis of standard histopathology and may enable discovery of novel spatial biomarkers for precision oncology.