Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Translating AI-Based Biopsy Sparing into Clinically Actionable Decision-Making for BI-RADS 4A+ Breast Lesions.

Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine·2026
Same author

Multi-omics Signature Predicts Anti-EGFR Therapy Benefit in Colorectal Cancer Liver Metastases: A Multi-center Cohort Study.

Current cancer drug targets·2026
Same author

Modulating tris-acid integrated proton-confined motifs for efficient uranium extraction from natural seawater.

Materials horizons·2026
Same author

Immune cell landscape reveals 5 immune-related subtypes and molecular characteristics with prognostic and therapeutic implications in pan-cancer.

Journal of leukocyte biology·2026
Same author

Glymphatic system metrics derived from DTI-ALPS are associated with cognitive impairment, brain atrophy, and plasma tauopathy biomarkers of type 2 diabetes patients: Analysis in dual-cohort.

Journal of Alzheimer's disease : JAD·2026
Same author

Financial Relief and Health Effects of Urban-Rural Health Insurance Integration on Older Rural Adults: A Causal Analysis of Age-Based Heterogeneity.

Healthcare (Basel, Switzerland)·2026

Related Experiment Video

Updated: Jul 15, 2026

Real-time Electrophysiology: Using Closed-loop Protocols to Probe Neuronal Dynamics and Beyond
08:08

Real-time Electrophysiology: Using Closed-loop Protocols to Probe Neuronal Dynamics and Beyond

Published on: June 24, 2015

MT-RevSNN: Memory and time efficient reversible framework for hierarchical spiking transformer.

Sirui Li1, Zeyang Song2, Xueyi Zhang1

  • 1School of Data Science, Chinese University of Hong Kong, Shenzhen, 518172, Shenzhen, China.

Neural Networks : the Official Journal of the International Neural Network Society
|July 13, 2026
PubMed
Summary

This study introduces MT-RevSNN, a novel reversible framework for Spiking Transformers. It significantly reduces memory and time costs for training hierarchical models, enabling efficient neuromorphic computing.

Keywords:
Memory efficiencyNeuromorphic computingReversible architectureSpiking neural networkSpiking transformer

More Related Videos

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
11:18

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks

Published on: March 2, 2015

Hybrid Microdrive System with Recoverable Opto-Silicon Probe and Tetrode for Dual-Site High Density Recording in Freely Moving Mice
08:57

Hybrid Microdrive System with Recoverable Opto-Silicon Probe and Tetrode for Dual-Site High Density Recording in Freely Moving Mice

Published on: August 10, 2019

Related Experiment Videos

Last Updated: Jul 15, 2026

Real-time Electrophysiology: Using Closed-loop Protocols to Probe Neuronal Dynamics and Beyond
08:08

Real-time Electrophysiology: Using Closed-loop Protocols to Probe Neuronal Dynamics and Beyond

Published on: June 24, 2015

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
11:18

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks

Published on: March 2, 2015

Hybrid Microdrive System with Recoverable Opto-Silicon Probe and Tetrode for Dual-Site High Density Recording in Freely Moving Mice
08:57

Hybrid Microdrive System with Recoverable Opto-Silicon Probe and Tetrode for Dual-Site High Density Recording in Freely Moving Mice

Published on: August 10, 2019

Area of Science:

  • Neuromorphic Computing
  • Artificial Intelligence
  • Deep Learning Architectures

Background:

  • Spiking Transformers offer energy-efficient neuromorphic computing but demand substantial memory (O(L×T)) for Spatio-Temporal Backpropagation (STBP).
  • Existing reversible architectures struggle with hierarchical spiking Transformers, lacking support for dimension rescaling and compatibility with integer spiking neurons, causing gradient instability.
  • Scaling neuromorphic computing with Spiking Transformers is hindered by memory and stability challenges in current reversible methods.

Purpose of the Study:

  • To introduce MT-RevSNN, the first reversible framework enabling O(1) memory for hierarchical spiking Transformers.
  • To overcome limitations of existing reversible methods in supporting dimension rescaling and integer spiking neurons.
  • To facilitate scalable and memory-efficient training of large-scale spiking Transformer models.

Main Methods:

  • Developed MT-RevSNN, a reversible framework for hierarchical spiking Transformers.
  • Introduced Information Fusion Blocks and Dual-Factor Scaling (DFS) for reversible dimension rescaling and stabilizing integer neuron dynamics.
  • Proposed Low-Rank Compressed Spiking Self-Attention (LRC-SSA) for lightweight implementation.

Main Results:

  • MT-RevSNN achieved O(1) memory footprint for hierarchical spiking Transformers.
  • Demonstrated significant reductions in training memory (12.4×) and training time (1.9×) on ImageNet-1K when implemented on QKFormer.
  • Maintained comparable accuracy to non-reversible QKFormer counterparts.

Conclusions:

  • MT-RevSNN effectively addresses memory and stability issues in training hierarchical spiking Transformers.
  • The proposed framework enables scalable, memory-efficient training of large-scale neuromorphic computing models.
  • Reversible architectures show significant potential for advancing efficient Spiking Transformer applications.