Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Per-Unit Sequence Models01:26

Per-Unit Sequence Models

An ideal Y-Y transformer, grounded through neutral impedances, displays per-unit sequence networks akin to those of a single-phase ideal transformer when subjected to balanced positive- or negative-sequence currents. These currents do not produce neutral currents, and their associated voltage drops.
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
Transformers in Distribution System01:27

Transformers in Distribution System

Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Reconstruction of Signal using Interpolation01:10

Reconstruction of Signal using Interpolation

Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next sampling...
Transformers with Off-Nominal Turns Ratios01:25

Transformers with Off-Nominal Turns Ratios

In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the rated...
Types Of Transformers01:16

Types Of Transformers

Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
Transformers01:26

Transformers

A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Training Together, Diagnosing Better: Federated Learning for Collagen VI-Related Dystrophies.

ArXiv·2026
Same author

FedADMM-InSa: An inexact and self-adaptive ADMM for federated learning.

Neural networks : the official journal of the International Neural Network Society·2024
Same author

Interplay between depth and width for interpolation in neural ODEs.

Neural networks : the official journal of the International Neural Network Society·2024
Same author

The background method: theory and computations.

Philosophical transactions. Series A, Mathematical, physical, and engineering sciences·2022
Same author

Linear projection-based chemical exchange saturation transfer parameter estimation.

NMR in biomedicine·2022

Related Experiment Videos

Exact sequence interpolation with transformers.

Albert Alcalde1, Giovanni Fantuzzi1, Enrique Zuazua1,2,3

  • 1Chair for Dynamics, Control, Machine Learning, and Numerics (Alexander von Humboldt Professorship), Friedrich-Alexander-Universität Erlangen-Nürnberg (1041), Cauerstrasse, 11, 91058 Erlangen, Germany.

Mathematical Foundations of Machine Learning
|May 25, 2026
PubMed
Summary

Transformers can precisely interpolate finite sequence datasets. This study constructs a transformer model with complexity independent of input length, offering convergence guarantees for learning tasks.

Keywords:
ApproximationDeep learningMathematics of machine learningSelf-attentionTransformer architecture

Related Experiment Videos

Area of Science:

  • Machine Learning
  • Deep Learning Theory
  • Sequence Modeling

Background:

  • Transformers excel at sequence-to-sequence tasks.
  • Understanding their interpolation capabilities is crucial for theoretical analysis.
  • Existing research often focuses on generalization rather than exact interpolation.

Purpose of the Study:

  • To prove transformers can exactly interpolate finite datasets.
  • To construct an efficient transformer architecture for exact interpolation.
  • To analyze the theoretical underpinnings of transformer interpolation.

Main Methods:

  • Constructing a transformer with alternating feed-forward and self-attention layers.
  • Utilizing low-rank parameter matrices in the self-attention mechanism.
  • Analyzing the model in both hardmax and softmax self-attention settings.

Main Results:

  • Demonstrated exact interpolation of finite sequence datasets in R^d.
  • Developed a transformer with O(sum(m^j)) blocks and O(d*sum(m^j)) parameters.
  • Achieved complexity independent of input sequence length.
  • Provided convergence guarantees for learning problems under regularized training.

Conclusions:

  • Transformers possess the capability for exact data interpolation.
  • The proposed construction offers theoretical insights into transformer efficiency.
  • This work bridges the gap between theoretical understanding and practical performance in sequence modeling.