A LLM-Based Hybrid-Transformer Diagnosis System in Healthcare

Area of Science:

• Medical Image Analysis
• Artificial Intelligence in Healthcare
• Computer Vision

Background:

• Traditional Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) often struggle with detailed information loss in medical imaging.
• Existing models face limitations in capturing global semantic information and rely on deep encoders with aggressive downsampling.

Purpose of the Study:

• To introduce a novel Large Language Model (LLM)-based Hybrid-Transformer Network (HybridTransNet) for enhanced medical image diagnosis.
• To address the limitations of existing architectures in preserving local and global context information.

Main Methods:

• Developed HybridTransNet, an LLM-based network encoding Big Data patches using a transformer mechanism.
• Implemented inter-scale and intra-scale self-attention within a transformer-based symmetric architecture.
• Incorporated a refining module and a novel fuzzy selector for improved feature recovery and output refinement.

Main Results:

• HybridTransNet demonstrated superior performance in brain tumor diagnosis compared to existing methods on two distinct datasets.
• The LLM-based approach effectively recovers both local and global context information crucial for accurate diagnosis.

Conclusions:

• The proposed LLM-based HybridTransNet offers a significant advancement in medical image analysis, particularly for brain tumor diagnosis.
• This novel architecture provides a robust solution for healthcare informatics, improving diagnostic accuracy through advanced AI techniques.