OPHash: learning of organ and pathology context-sensitive hashing for medical image retrieval

Retrieving images of organs and their associated pathologies is essential for evidence-based clinical diagnosis. Deep neural hashing (DNH) has demonstrated the ability to retrieve images fast on large datasets. Conventional pairwise DNH methods can focus on semantic similarity between either organs or pathology of an image pair but not on both simultaneously.

We propose an organ and pathology contextual-supervised hashing approach (OPHash) learned using three types of samples (called bags) to learn accurate hash representation. Because only semantic similarity is inadequate to incorporate with these bags, we introduce relational similarity to generate identical hash codes from most similar image pairs. OPHash is trained by minimizing classification loss, two retrieval losses implemented using Cauchy cross-entropy and maximizing discriminator loss over training samples.

Experiments are performed with two radiology datasets derived from the publicly available datasets. OPHash achieves 24% higher mean average precision than the state-of-the-art for top-100 retrieval.

OPHash retrieves images with semantic similarity of organs and their associated pathology. It is agnostic to image size as well. This method improves retrieval efficiency across diverse medical imaging datasets, accommodating multiple organs and pathologies. The code is available at https://github.com/asimmanna17/OPHash.