Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Palladium-catalyzed olefination and arylation of 2-substituted 1,2,3-triazole N-oxides.

Organic letters·2013
Same author

One-stop hybrid coronary revascularization versus coronary artery bypass grafting and percutaneous coronary intervention for the treatment of multivessel coronary artery disease: 3-year follow-up results from a single institution.

Journal of the American College of Cardiology·2013
Same author

Relative contributions of the thalamus and the paraventricular nucleus of the hypothalamus to the cardiac sympathetic afferent reflex.

American journal of physiology. Regulatory, integrative and comparative physiology·2013
Same author

Initial light soaking treatment enables hole transport material to outperform spiro-OMeTAD in solid-state dye-sensitized solar cells.

Journal of the American Chemical Society·2013
Same author

The rice GERMINATION DEFECTIVE 1, encoding a B3 domain transcriptional repressor, regulates seed germination and seedling development by integrating GA and carbohydrate metabolism.

The Plant journal : for cell and molecular biology·2013
Same author

Salvage intensity modulated radiotherapy using endorectal balloon after radical prostatectomy: clinical outcomes.

International journal of urology : official journal of the Japanese Urological Association·2013

Related Experiment Video

Updated: Jun 14, 2025

P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation
06:09

P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

Published on: September 8, 2023

526

SNN-BERT: Training-efficient Spiking Neural Networks for energy-efficient BERT.

Qiaoyi Su1, Shijie Mei2, Xingrun Xing2

  • 1School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China; Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China.

Neural Networks : the Official Journal of the International Neural Network Society
|August 29, 2024
PubMed
Summary

Spiking Neural Networks (SNNs) now use "individual coding" for better temporal processing in NLP tasks. This new approach, with Bidirectional Parallel Spiking Neurons (BPSNs), significantly cuts energy use and improves performance.

Keywords:
Energy efficiencyNatural language processingSpiking neural networkTraining efficiency

More Related Videos

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
11:18

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks

Published on: March 2, 2015

10.3K
A Simple Stimulatory Device for Evoking Point-like Tactile Stimuli: A Searchlight for LFP to Spike Transitions
07:34

A Simple Stimulatory Device for Evoking Point-like Tactile Stimuli: A Searchlight for LFP to Spike Transitions

Published on: March 25, 2014

9.9K

Related Experiment Videos

Last Updated: Jun 14, 2025

P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation
06:09

P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

Published on: September 8, 2023

526
Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
11:18

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks

Published on: March 2, 2015

10.3K
A Simple Stimulatory Device for Evoking Point-like Tactile Stimuli: A Searchlight for LFP to Spike Transitions
07:34

A Simple Stimulatory Device for Evoking Point-like Tactile Stimuli: A Searchlight for LFP to Spike Transitions

Published on: March 25, 2014

9.9K

Area of Science:

  • Neuromorphic computing
  • Artificial intelligence
  • Natural Language Processing

Background:

  • Spiking Neural Networks (SNNs) offer low-power processing for sequence tasks due to their brain-inspired dynamics.
  • Current SNNs use inefficient "repeat coding," limiting temporal understanding and increasing memory load.
  • This limits the potential of SNNs in complex NLP applications.

Purpose of the Study:

  • To introduce an improved input coding strategy called "individual coding" for SNNs.
  • To develop a Bidirectional Parallel Spiking Neuron (BPSN) to address the increased training time associated with individual coding.
  • To enhance the temporal modeling capabilities and efficiency of SNNs for NLP tasks.

Main Methods:

  • Implemented "individual coding" by aligning input tokens with timesteps.
  • Designed a Bidirectional Parallel Spiking Neuron (BPSN) supporting parallel computation and adaptive sequence lengths.
  • Developed SNN-BERT, a deep direct training SNN architecture based on BERT, utilizing the BPSN.
  • Validated the approach on the GLUE dataset for NLP tasks.

Main Results:

  • Achieved a 6.46× reduction in energy consumption compared to repeat coding baselines.
  • Improved performance by 16.1%, reaching 74.4% on the GLUE dataset.
  • Demonstrated 3.5× training acceleration and 3.8× memory optimization.
  • Obtained comparable performance to ANNs with up to 22.5× greater energy efficiency.

Conclusions:

  • "Individual coding" combined with BPSNs offers a more efficient and effective approach for SNNs in NLP.
  • The SNN-BERT model pushes the performance boundaries for SNNs on benchmark NLP tasks.
  • This work highlights the potential of SNNs for energy-efficient and high-performance AI.