CAM-Interacted Vision GNN for Multi-Label Medical Images

Area of Science:

• Computer Vision
• Artificial Intelligence
• Medical Imaging Analysis

Background:

• Vision Graph Neural Networks (ViG) process images using graph-level analysis.
• ViG's reliance on appearance-level neighbors overlooks category semantics, hindering multi-label medical image learning.
• Pixel-level annotations for medical images are scarce, preventing direct construction of category-aware graphs.

Purpose of the Study:

• To propose a novel method, CAM-interacted Vision GNN (CiV-GNN), for improved multi-label medical image learning.
• To address the limitations of ViG by incorporating category semantics without manual pixel-level annotations.
• To enhance the distinctiveness of categories in medical image analysis.

Main Methods:

• Utilizing Class Activation Maps (CAMs) to localize category-specific regions without manual annotations.
• Introducing a Class-activated Patch Division (CAPD) module for category-aware graph construction guided by CAMs.
• Developing a Multi-graph Interactive Processing (MIP) module to model inter-graph relations and promote inter-category learning.

Main Results:

• CiV-GNN demonstrates strong performance in surgical tool localization and multi-label medical image classification.
• Achieved a 1.43% improvement in mAP50 and a 7.02% improvement in mAP50-95 on the m2cai16-localization dataset compared to YOLOv8.
• Effectively integrates category semantics into graph construction for enhanced image analysis.

Conclusions:

• CiV-GNN successfully overcomes the limitations of traditional ViG by incorporating category information through CAMs.
• The proposed method offers a viable solution for multi-label medical image learning, particularly when pixel-level annotations are unavailable.
• CiV-GNN shows significant potential for improving diagnostic accuracy and localization in medical imaging applications.