Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Dot Product01:29

Dot Product

760
The dot product is an essential concept in mathematics and physics.
In engineering, the dot product of any two vectors is the product of the magnitudes of the vectors and the cosine of the angle between them. It is denoted by a dot symbol between the two vectors.
Consider a vehicle pulling an object along the ground using a rope. If the rope makes an angle with the horizontal axis, the work done can be calculated using the dot product of the force applied and the object's displacement.
The dot...
760
The Dot Product01:26

The Dot Product

149
Measuring how one directional quantity affects another along a specific path involves comparing their orientation and strength. When two such quantities are represented using direction and amount, a numerical result is computed to show how much one acts along the path of the other. This result comes from a rule combining both inputs' horizontal and vertical parts and adding the results.This calculation gives a single value that grows larger when both inputs point in similar directions and...
149
Dot Product: Problem Solving01:21

Dot Product: Problem Solving

655
The dot product is a powerful tool in problem-solving involving vectors, given that the dot product of two vectors is the product of their magnitudes and the cosine of the angle between them measured anti-clockwise. Solving problems involving the dot product requires understanding its properties and developing a step-by-step process to solve them. Here are the main steps to follow when solving any general problem involving the dot product:
Identify the problem: Start by reading the problem and...
655
Scalar Product (Dot Product)01:11

Scalar Product (Dot Product)

25.9K
The scalar multiplication of two vectors is known as the scalar or dot product. As the name indicates, the scalar product of two vectors results in a number, that is, a scalar quantity. Scalar products are used to define work and energy relations. For example, the work that a force (a vector) performs on an object while causing its displacement (a vector) is defined as a scalar product of the force vector with the displacement vector.
The scalar product of two vectors is obtained by multiplying...
25.9K
Reducing Line Loss01:18

Reducing Line Loss

338
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss in...
338
Mean Absolute Deviation01:13

Mean Absolute Deviation

3.2K
The mean absolute deviation is also a measure of the variability of data in a sample. It is the absolute value of the average difference between the data values and the mean.
Let us consider a dataset containing the number of unsold cupcakes in five shops: 10, 15, 8, 7, and 10. Initially, calculate the sample mean. Then calculate the deviation, or the difference, between each data value and the mean. Next, the absolute values of these deviations are added and divided by the sample size to...
3.2K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Impact of labelling inaccuracy and image noise on tooth segmentation in panoramic radiographs using federated, centralized, and local learning.

Dento maxillo facial radiology·2026
Same author

A multimodal stress detection dataset with facial expressions and physiological signals.

Scientific data·2025
Same author

Manifold Gaussian Variational Bayes on the Precision Matrix.

Neural computation·2024
Same author

A synthetic data set to benchmark anti-money laundering methods.

Scientific data·2023
Same author

Automatic social distance estimation for photographic studies: Performance evaluation, test benchmark, and algorithm.

Machine learning with applications·2022
Same author

Single-layer vision transformers for more accurate early exits with less overhead.

Neural networks : the official journal of the International Neural Network Society·2022

Related Experiment Video

Updated: Jan 7, 2026

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications
03:31

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

990

Continual low-rank scaled dot-product attention.

Ginés Carreto Picón1, Illia Oleksiienko1, Lukas Hedegaard1

  • 1Department of Electrical and Computer Engineering, Aarhus University, Finlandsgade 22, Aarhus N, 8200, Denmark.

Neural Networks : the Official Journal of the International Neural Network Society
|December 31, 2025
PubMed
Summary

This study introduces a Nyström approximation for Scaled Dot-product Attention, significantly reducing computational costs for stream data processing. The new method enables efficient continual inference in Transformers without sacrificing performance.

Keywords:
Continual inferenceDeep learningNyström approximationScaled dot-product attentionTransformer

More Related Videos

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique
04:48

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique

Published on: July 5, 2024

721

Related Experiment Videos

Last Updated: Jan 7, 2026

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications
03:31

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

990
Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique
04:48

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique

Published on: July 5, 2024

721

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Deep Learning

Background:

  • Transformers excel at sequence processing for static tasks.
  • The Scaled Dot-product Attention in Transformers has high computational and memory demands, limiting its use in real-time stream data processing.
  • Existing methods to reduce Transformer costs include low-rank approximations and sparsity.

Purpose of the Study:

  • To develop an efficient formulation of Scaled Dot-product Attention suitable for continual inference.
  • To address the computational and memory constraints of Transformers in stream data applications.

Main Methods:

  • Introduced a novel formulation of Scaled Dot-product Attention utilizing the Nyström approximation.
  • Developed a method for efficient continual inference with Transformers.

Main Results:

  • The proposed Continual Scaled Dot-product Attention reduced operations by up to three orders of magnitude compared to original Transformers.
  • Maintained predictive performance comparable to existing models in online tasks.

Conclusions:

  • The Nyström approximation offers a computationally efficient alternative for Scaled Dot-product Attention in stream data scenarios.
  • This approach makes Transformers more feasible for real-time applications with resource constraints.