A novel cost function for nuclei segmentation and classification in imbalanced histopathology data-sets

Cancer is a major global health problem, causing millions of deaths yearly. Histopathological analysis plays a crucial role in detecting and diagnosing various types of cancer, enabling an accurate diagnosis to inform targeted treatment planning, allowing for better cancer staging, and ultimately improving prognosis. We aim to detect cancer earlier, which can ultimately help reduce mortality rates and enhance patients' quality of life. However, detecting and classifying rare cells is a key challenge for pathologists and researchers. Many histopathological data-sets contain imbalanced data, with only a few instances of rare cells whose unique morphological structures can impede early diagnosis efforts. Our model, SPNet, a spatially aware convolutional neural network, addresses this problem by employing a spatial data balancing technique, enhancing the classification of rare nuclei by 21.8 %. Since nuclei often cluster and exhibit patterns of the same class, SPNet's novel cost function targets spatial regions, resulting in a 1.9 % increase in the F₁ classification of rare class types within the CoNSeP dataset. When integrated with a ResNet50-SE encoder, SPNet increases the mean F₁ score for classifying all nuclei in the CoNSeP dataset by 4.3 %, compared to the benchmark set by the state-of-the-art HoVer-Net model. The potential integration of SPNet into existing medical devices could allow us to streamline diagnostic processes and minimise false negatives.