Medical hierarchical image classification via dual-geometry image-text learning

Area of Science:

• Medical Image Analysis
• Computer Vision
• Machine Learning

Background:

• Hierarchical image classification is crucial for medical image analysis, reflecting biological and clinical relationships.
• Current methods often involve complex model designs and training strategies for multi-task learning and fine-grained detection.

Purpose of the Study:

• To exploit the negative curvature of hyperbolic space for efficient representation of hierarchical structures in medical images.
• To propose a novel dual-geometry image-text framework (H2CL) for improved hierarchical image classification.

Main Methods:

• Developed H2CL, a framework utilizing a lightweight classifier head to extract both Euclidean and hyperbolic features from images.
• Integrated a text branch with entailment loss to model image-text alignment and inter-sample relationships.
• Employed hyperbolic space properties for efficient representation of hierarchical data.

Main Results:

• H2CL consistently outperformed advanced methods on cervical cell, skin lesion, and gallbladder disease datasets.
• Achieved an average accuracy improvement of 7% at the fine-grained level compared to standard Swin Transformer.
• Demonstrated consistent performance gains when integrated with various backbone models.

Conclusions:

• The proposed H2CL framework effectively leverages dual-geometry representations for enhanced hierarchical image classification.
• The integration of image and text features, guided by hyperbolic geometry, offers a promising direction for medical image analysis.
• H2CL provides a robust and efficient solution for complex medical image classification tasks.