Lightweight plant phenotypic feature extraction via transferable attention head pruning in Vision Transformers

Area of Science:

• Computer Vision
• Machine Learning
• Plant Science

Background:

• Vision Transformers (ViTs) face challenges with high dimensionality in attention mechanisms, leading to redundant computations.
• Generalizing plant phenotypic feature extraction across diverse species requires efficient models that minimize computational load.

Purpose of the Study:

• To propose a lightweight Multi-Head Self-Attention (MHSA) mechanism for efficient plant phenotypic feature extraction.
• To improve model efficiency and generalization capabilities across plant species using transfer learning and dynamic head pruning.

Main Methods:

• Developed the Transferable Attention Head Alignment (TAHA) framework involving pre-training, cross-species alignment (Domain Alignment Loss), and head pruning based on transferability scores.
• Integrated pruned MHSA modules with standard Transformer backbones for deployment on edge devices.
• Conducted experiments on edge hardware (Raspberry Pi 4, NVIDIA Jetson Nano) and GPUs.

Main Results:

• Achieved accuracy comparable to full-head models while reducing computational expenses by up to 40% (e.g., 14.1 ms inference latency on Raspberry Pi 4).
• Demonstrated efficient deployment on resource-constrained edge devices.
• Identified generalization limits with a 7.2% F1-score drop for phylogenetically distant species transfer (Arabidopsis → pine).

Conclusions:

• The proposed method offers a balanced compromise between performance and efficiency for plant phenotyping.
• Enables scalable plant phenotyping in computationally constrained environments.
• Facilitates practical agricultural applications through gradual adaptation to novel species without complete retraining.