Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Modeling with Differential Equations01:25

Modeling with Differential Equations

114
Population dynamics can be described mathematically by considering the population size P(t) as a function of time. The rate of change of the population is then represented by the derivative of P(t). A simple assumption is that the rate of growth is proportional to the size of the population itself. This leads to an exponential growth model, where the population increases rapidly without bound. While this is a useful first approximation, it does not reflect realistic long-term...
114
State Space Representation01:27

State Space Representation

625
The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
625
Linear Approximation in Time Domain01:21

Linear Approximation in Time Domain

382
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
382
Language Development01:22

Language Development

977
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
977
Transfer Function to State Space01:23

Transfer Function to State Space

834
State-space representation is a powerful tool for simulating physical systems on digital computers, necessitating the conversion of the transfer function into state-space form. Consider an nth-order linear differential equation with constant coefficients, like those encountered in an RLC circuit. The state variables are selected as the output and its n−1 derivatives. Differentiating these variables and substituting them back into the original equation produces the state equations.
In an RLC...
834
Introduction to Learning01:18

Introduction to Learning

1.3K
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
1.3K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

AI Is Changing the Landscape of Academic Writing: What Can Be Done? Authors' Reply to: AI Increases the Pressure to Overhaul the Scientific Peer Review Process. Comment on "Artificial Intelligence Can Generate Fraudulent but Authentic-Looking Scientific Medical Articles: Pandora's Box Has Been Opened".

Journal of medical Internet research·2023
Same author

Emergence of Self-Reproducing Metabolisms as Recursive Algorithms in an Artificial Chemistry.

Artificial life·2022
Same author

Classification of Discrete Dynamical Systems Based on Transients.

Artificial life·2021
Same author

Holographic Declarative Memory: Distributional Semantics as the Architecture of Memory.

Cognitive science·2020
Same author

An Empirical Evaluation of Rule Extraction from Recurrent Neural Networks.

Neural computation·2018
Same author

Integrated, Not Isolated: Defining Typological Proximity in an Integrated Multilingual Architecture.

Frontiers in psychology·2018

Related Experiment Video

Updated: Feb 22, 2026

Constructing and Visualizing Models using Mime-based Machine-learning Framework
06:19

Constructing and Visualizing Models using Mime-based Machine-learning Framework

Published on: July 22, 2025

2.6K

Learning Simpler Language Models with the Differential State Framework.

Alexander G Ororbia Ii1, Tomas Mikolov2, David Reitter3

  • 1College of Information Sciences and Technology, Pennsylvania State University, State College, PA 16802, U.S.A. ago109@ist.psu.edu.

Neural Computation
|September 29, 2017
PubMed
Summary

The delta-RNN, a new neural network model, efficiently learns from long time lags in language modeling. It outperforms complex models like LSTM and GRU with fewer parameters.

More Related Videos

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

8.1K
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.2K

Related Experiment Videos

Last Updated: Feb 22, 2026

Constructing and Visualizing Models using Mime-based Machine-learning Framework
06:19

Constructing and Visualizing Models using Mime-based Machine-learning Framework

Published on: July 22, 2025

2.6K
Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

8.1K
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

1.2K

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Deep Learning

Background:

  • Temporal neural models struggle with learning information across long time lags.
  • Existing complex architectures are computationally expensive to train.
  • The differential state framework (DSF) offers a unified and efficient approach.

Purpose of the Study:

  • Introduce a novel neural network architecture, the delta-RNN, within the DSF framework.
  • Evaluate the delta-RNN's performance in language modeling tasks.
  • Compare the delta-RNN against established complex architectures.

Main Methods:

  • Developed the delta-RNN, a model requiring minimal additional parameters compared to simple recurrent networks.
  • Utilized the differential state framework (DSF) for interpolating between fast and slow states.
  • Evaluated performance on word, character, and subword level language modeling.

Main Results:

  • The delta-RNN significantly outperforms Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models in word and character level language modeling.
  • When regularized, the delta-RNN achieves performance comparable to state-of-the-art baselines.
  • At the subword level, delta-RNN performance is on par with complex gated architectures.

Conclusions:

  • The delta-RNN provides a simple yet high-performing solution for learning across long time lags.
  • This architecture offers an efficient alternative to complex models for temporal data processing.
  • The delta-RNN demonstrates strong potential for advancing language modeling and related sequence tasks.