How Does Attention Work in Vision Transformers? A Visual Analytics Attempt.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Vision Transformers (ViTs) apply transformer models to image analysis by dividing images into patches.
• Self-attention mechanisms in ViTs learn relationships between these patches.
• Interpreting ViT behavior, particularly attention head importance and patterns, remains a challenge.

Purpose of the Study:

• To develop a visual analytics approach for interpreting Vision Transformers (ViTs).
• To answer key questions regarding attention head importance, spatial attention, and learned attention patterns within ViTs.
• To deepen the understanding of how ViTs process visual information.

Main Methods:

• Introduced pruning-based metrics to identify important attention heads in ViTs.
• Profiled spatial distribution and layer-wise trends of attention strengths between image patches.
• Employed an autoencoder-based solution to summarize learned attention patterns across heads.
• Conducted case studies with deep learning experts to validate the approach.

Main Results:

• Quantified the importance of individual attention heads within ViTs.
• Visualized attention strength distributions, revealing spatial relationships and layer-wise dynamics.
• Characterized distinct attention patterns learned by different heads.
• Demonstrated that important heads exhibit specific attention strengths and patterns.

Conclusions:

• The visual analytics approach effectively deepens the understanding of Vision Transformer mechanisms.
• Key insights were gained into head importance, attention strength, and attention patterns in ViTs.
• This work provides a framework for interpreting complex deep learning models in computer vision.