Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Universality of shocks in conserved driven single-file motions with bottlenecks.

Physical review. E·2026

Same author

Stationary densities and delocalized domain walls in asymmetric exclusion processes competing for finite pools of resources.

Physical review. E·2026

Same author

Insights Into Virus-Encoded RNA Silencing Suppressors Across Viral Families: A Focus on Viruses Infecting Solanaceae Crops.

Physiologia plantarum·2026

Same author

Immunotherapy in triple-negative breast cancer: From molecular mechanisms to precision medicine-overcoming resistance and optimizing clinical outcomes.

Critical reviews in oncology/hematology·2026

Same author

Mixed fungal infection in immunocompetent Dengue patient.

Indian journal of medical microbiology·2026

Same author

Homogenous Catalysis for Ammonia-Borane Dehydrogenation by Transition Metal-Based Complexes: A Mini Review.

Chemistry, an Asian journal·2026

Same journal

What do LLMs value? An evaluation framework for revealing subjective trade-offs in assessment of glycemic control.

Proceedings of machine learning research·2026

Same journal

Towards the Efficient Inference by Incorporating Automated Computational Phenotypes under Covariate Shift.

Proceedings of machine learning research·2026

Same journal

Endo-SemiS: Towards Robust Semi-Supervised Image Segmentation for Endoscopic Video.

Proceedings of machine learning research·2026

Same journal

Perspective: Machine Learning for Health Should Consider Social Drivers of Health.

Proceedings of machine learning research·2026

Same journal

Classifying Phonotrauma Severity from Vocal Fold Images with Soft Ordinal Regression.

Proceedings of machine learning research·2026

Same journal

Does Domain-Specific Retrieval Augmented Generation Help LLMs Answer Consumer Health Questions?

Proceedings of machine learning research·2026

See all related articles

Search research articles

Related Experiment Video

Updated: Jul 31, 2025

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Published on: December 15, 2023

Multi Resolution Analysis (MRA) for Approximate Self-Attention.

Zhanpeng Zeng¹, Sourav Pal¹, Jeffery Kline²

¹University of Wisconsin, Madison, USA.

Proceedings of Machine Learning Research

|May 4, 2023

Summary

This summary is machine-generated.

This study introduces a novel Multiresolution Analysis (MRA) approach for efficient Transformer self-attention. The MRA-based method, inspired by Wavelets, outperforms existing efficient attention mechanisms for various sequence lengths.

More Related Videos

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique

Published on: July 5, 2024

Author Spotlight: Advancing Large-Scale Neural Dynamics Through HD-MEA Technology

Author Spotlight: Advancing Large-Scale Neural Dynamics Through HD-MEA Technology

Published on: March 8, 2024

Related Experiment Videos

Last Updated: Jul 31, 2025

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Published on: December 15, 2023

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique

Published on: July 5, 2024

Author Spotlight: Advancing Large-Scale Neural Dynamics Through HD-MEA Technology

Author Spotlight: Advancing Large-Scale Neural Dynamics Through HD-MEA Technology

Published on: March 8, 2024

Area of Science:

Artificial Intelligence
Machine Learning
Computer Science

Background:

Transformers are dominant models in NLP and vision.
Efficient training and deployment of Transformers are crucial.
Approximating the self-attention matrix is key to efficiency.

Purpose of the Study:

To explore the potential of Multiresolution Analysis (MRA) for efficient Transformer self-attention.
To develop an MRA-based self-attention mechanism.
To evaluate its performance against existing efficient methods.

Main Methods:

Revisiting classical Multiresolution Analysis (MRA) concepts, specifically Wavelets.
Developing approximations based on empirical feedback and modern hardware considerations.
Extensive experimental evaluation of the proposed MRA-based self-attention.

Main Results:

The proposed MRA-based self-attention achieves an excellent performance profile.
This multi-resolution scheme outperforms most existing efficient self-attention proposals.
The method is effective for both short and long sequences.

Conclusions:

Multiresolution Analysis offers a promising avenue for efficient Transformer self-attention.
The developed MRA approach provides a competitive and efficient alternative.
This work highlights the underexplored potential of Wavelet-inspired methods in deep learning architectures.