Hierarchical Contrastive Learning for Precise Whole-Body Anatomical Localization in PET/CT Imaging

Area of Science:

• Medical Imaging
• Radiology
• Artificial Intelligence

Background:

• Accurate anatomical localization is crucial for radiology report generation but remains challenging.
• Segmentation-based methods often fail in severe disease due to low accuracy.
• Lesion localization has received less attention compared to detection and segmentation.

Purpose of the Study:

• To reformulate anatomical localization as an image-to-text retrieval task.
• To develop a robust framework for fine-grained lesion localization across the entire body.
• To improve the accuracy and efficiency of anatomical localization in radiology.

Main Methods:

• Proposed a CLIP-based framework aligning lesion image patches with anatomical text descriptions.
• Implemented hierarchical anatomical retrieval (387 locations, two-level hierarchy).
• Utilized augmented location descriptions and semi-hard negative sample mining for enhanced learning.

Main Results:

• Achieved 84.13% localization accuracy on internal test sets and 80.42% on external test sets.
• Demonstrated a per-lesion inference time of 34 ms.
• Showed superior robustness in complex clinical cases compared to segmentation-based approaches.

Conclusions:

• The proposed image-to-text retrieval framework significantly enhances anatomical localization accuracy.
• The method offers a robust and efficient solution for lesion localization in radiology.
• This approach has the potential to improve the quality and consistency of radiology reports.