Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Linear Approximation in Frequency Domain01:26

Linear Approximation in Frequency Domain

146
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
146
Current Growth And Decay In RL Circuits01:30

Current Growth And Decay In RL Circuits

4.1K
The current growth and decay in RL circuits can be understood by considering a series RL circuit consisting of a resistor, an inductor, a constant source of emf, and two switches. When the first switch is closed, the circuit is equivalent to a single-loop circuit consisting of a resistor and an inductor connected to a source of emf. In this case, the source of emf produces a current in the circuit. If there were no self-inductance in the circuit, the current would rise immediately to a steady...
4.1K
Gradient and Del Operator01:14

Gradient and Del Operator

3.1K
In mathematics and physics, the gradient and del operator are fundamental concepts used to describe the behavior of functions and fields in space. The gradient is a mathematical operator that gives both the magnitude and direction of the maximum spatial rate of change. Consider a person standing on a mountain. The slope of the mountain at any given point is not defined unless it is quantified in a particular direction. For this reason, a "directional derivative" is defined, which is a vector...
3.1K
Underflow Gates01:30

Underflow Gates

109
Underflow gates are vital for controlling water flow in irrigation canals. The three main types of underflow gates — vertical, radial, and drum gates — serve different purposes while ensuring effective flow management. Vertical gates move up and down, generating a free-flowing water jet; radial gates pivot to regulate the flow; and drum gates rotate for precise adjustments. The flow through these gates is influenced by downstream conditions, resulting in free or drowned outflow.Free and...
109
RL Circuit with Source01:14

RL Circuit with Source

1.0K
When an RL (Resistor-Inductor) circuit is connected to a DC source, the complete response of the circuit can be divided into two parts: the transient response and the steady-state response.
The transient response of the circuit is its temporary reaction to the sudden application of the DC source. This response is characterized by a current that exponentially decays to zero as time approaches infinity. During this transitional period, the inductor behaves like a short circuit, causing the source...
1.0K
RL Circuits01:14

RL Circuits

2.6K
An RL circuit consists of a resistor and an inductor and may have a source of emf connected to it. The inductor in the circuit helps to prevent rapid changes in current, which can be helpful if a steady current is required but the external source has a fluctuating emf. Consider an open RL circuit connected to a source of constant emf. As soon as the circuit is closed, the current begins to increase at a rate that depends only on the value of the inductance in the circuit. The greater the...
2.6K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

An NLP-assisted Bayesian time-series analysis for prevalence of Twitter cyberbullying during the COVID-19 pandemic.

Social network analysis and mining·2023
Same author

A model-free approach to do long-term volatility forecasting and its variants.

Financial innovation·2023
See all related articles

Related Experiment Video

Updated: Sep 26, 2025

Inducing Long-Term Plasticity of Intrinsic Neuronal Excitability in Neurons of the Dorsal Lateral Geniculate Nucleus
05:01

Inducing Long-Term Plasticity of Intrinsic Neuronal Excitability in Neurons of the Dorsal Lateral Geniculate Nucleus

Published on: September 20, 2024

517

Provable training of a ReLU gate with an iterative non-gradient algorithm.

Sayar Karmakar1, Anirbit Mukherjee2

  • 1Department of Statistics, University of Florida, 230 Newell Drive, Gainesville, 32611, FL, U.S.A..

Neural Networks : the Official Journal of the International Neural Network Society
|April 22, 2022
PubMed
Summary

This study introduces a new algorithm for training ReLU gates, achieving linear time training under milder data conditions. It also offers robust recovery of true parameters against data poisoning attacks.

Keywords:
Neural netsNon-gradient iterative algorithmsNon-smooth non-convex optimizationStochastic algorithms

More Related Videos

Computational Modeling of Retinal Neurons for Visual Prosthesis Research - Fundamental Approaches
10:50

Computational Modeling of Retinal Neurons for Visual Prosthesis Research - Fundamental Approaches

Published on: June 21, 2022

1.8K
Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
11:18

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks

Published on: March 2, 2015

10.5K

Related Experiment Videos

Last Updated: Sep 26, 2025

Inducing Long-Term Plasticity of Intrinsic Neuronal Excitability in Neurons of the Dorsal Lateral Geniculate Nucleus
05:01

Inducing Long-Term Plasticity of Intrinsic Neuronal Excitability in Neurons of the Dorsal Lateral Geniculate Nucleus

Published on: September 20, 2024

517
Computational Modeling of Retinal Neurons for Visual Prosthesis Research - Fundamental Approaches
10:50

Computational Modeling of Retinal Neurons for Visual Prosthesis Research - Fundamental Approaches

Published on: June 21, 2022

1.8K
Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
11:18

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks

Published on: March 2, 2015

10.5K

Area of Science:

  • Machine Learning
  • Artificial Intelligence
  • Optimization Theory

Background:

  • Training single ReLU gates is fundamental in deep learning.
  • Existing methods often require strong data distribution assumptions or lack robustness.
  • Unexplored regimes for ReLU gate training necessitate novel algorithmic approaches.

Purpose of the Study:

  • To provide provable guarantees for training a single ReLU gate in new regimes.
  • To develop an algorithm resilient to data-poisoning attacks during training.
  • To analyze the impact of mini-batching on convergence time.

Main Methods:

  • A simple iterative stochastic algorithm is proposed.
  • Analysis incorporates moment assumptions for approximate parameter recovery.
  • Convergence analysis considers mini-batching and its scaling effects.

Main Results:

  • Linear time training of ReLU gates achieved under milder data conditions.
  • First-of-its-kind approximate recovery of parameters under data-poisoning attacks.
  • Near-optimal worst-case guarantees with graceful degradation under attack magnitude.

Conclusions:

  • The developed algorithm offers efficient and robust training for ReLU gates.
  • The findings advance theoretical understanding of neural network training.
  • Simulation results show similarities to Stochastic Gradient Descent (SGD).