High-Performance Method and Architecture for Attention Computation in DNN Inference.

Area of Science:

• Artificial Intelligence
• Computer Engineering
• Medical Imaging

Background:

• Deep learning and Attention mechanisms are crucial in medical imaging.
• Existing hardware architectures face challenges with resource consumption, accuracy, and efficient deployment for DNN accelerators.

Purpose of the Study:

• To propose an online-programmable Attention hardware architecture based on compute-in-memory (CIM).
• To reduce hardware complexity, improve integration density, energy efficiency, and calculation accuracy for Attention mechanisms.

Main Methods:

• Decomposing Attention computation into cascaded matrix operations.
• Designing an online-programmable CIM architecture with dynamic weight adjustment for improved accuracy.
• Verifying the architecture's applicability for DNN inference via Spice simulation.

Main Results:

• The proposed architecture significantly reduces hardware implementation complexity.
• Achieved substantial improvements in integration density and energy efficiency (over 91.38x).
• Demonstrated a 12.5x improvement in latency and computing efficiency compared to traditional architectures.

Conclusions:

• The developed Attention hardware architecture effectively addresses the limitations of existing systems.
• It offers a promising solution for efficient and accurate deployment of deep learning models in medical imaging.
• The CIM-based approach provides significant performance gains and resource optimization.