Hybrid Dual-Heterogeneous Knowledge Distillation Network for Anomaly Detection in Retinal OCT Images

Unsupervised medical anomaly detection aims to identify abnormal images by training exclusively on normal samples, thereby enabling the detection of disease-related irregularities without the need for large-scale labeled datasets. Current knowledge distillation-based methods typically detect anomalies by comparing feature discrepancies between teacher and student networks. However, because these methods employ an optimization strategy where the teacher and student architectures are highly similar, the student network's features tend to closely mirror those of the teacher, leading to an identity mapping issue. Moreover, the diversity of lesion types in retinal Optical Coherence Tomography (OCT) images further complicates anomaly detection. In this paper, we propose a novel hybrid dual-heterogeneous knowledge distillation network to overcome these challenges. Our approach consists of a teacher network with an encoder-only architecture and a student network that integrates an encoder with dual decoders. This heterogeneous design effectively mitigates the identity mapping problem, enhancing sensitivity to both structural and logical anomalies. Specifically, our Multi-Feature Model leverages convolutional and depthwise convolutional blocks to extract and integrate local features for structural anomaly detection, while the Mamba UpNet employs self-supervised learning to capture long-range dependencies and global anomaly patterns. Extensive experiments on two retinal OCT anomaly detection datasets demonstrate that our method achieves state-of-the-art performance, effectively handling diverse anomaly types.