Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Linear time-invariant Systems01:23

Linear time-invariant Systems

1.0K
A system is linear if it displays the characteristics of homogeneity and additivity, together termed the superposition property. This principle is fundamental in all linear systems. Linear time-invariant (LTI) systems include systems with linear elements and constant parameters.
The input-output behavior of an LTI system can be fully defined by its response to an impulsive excitation at its input. Once this impulse response is known, the system's reaction to any other input can be...
1.0K
BIBO stability of continuous and discrete -time systems01:24

BIBO stability of continuous and discrete -time systems

1.0K
System stability is a fundamental concept in signal processing, often assessed using convolution. For a system to be considered bounded-input bounded-output (BIBO) stable, any bounded input signal must produce a bounded output signal. A bounded input signal is one where the modulus does not exceed a certain constant at any point in time.
To determine the BIBO stability, the convolution integral is utilized when a bounded continuous-time input is applied to a Linear Time-Invariant (LTI) system....
1.0K
Linear Approximation in Frequency Domain01:26

Linear Approximation in Frequency Domain

418
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
418
Random Variables01:09

Random Variables

18.7K
A random variable is a single numerical value that indicates the outcome of a procedure. The concept of random variables is fundamental to the probability theory and was introduced by a Russian mathematician, Pafnuty Chebyshev, in the mid-nineteenth century.
Uppercase letters such as X or Y denote a random variable. Lowercase letters like x or y denote the value of a random variable. If X is a random variable, then X is written in words, and x is given as a number.
For example, let X = the...
18.7K
State Space Representation01:27

State Space Representation

668
The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
668
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

387
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
387

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

A pilot study of brain correlates of long-term treatment with transcutaneous vagal nerve stimulation in posttraumatic stress disorder.

Journal of affective disorders reportsยท2026
Same author

A Machine Learning Driven Approach to Quantifying Coronary Artery Tortuosity.

JACC. Advancesยท2026
Same author

Is More Always Better With Digital Health Interventions? Shifting Engagement From Maximizing Use to Supporting Health.

Mayo Clinic proceedings. Digital healthยท2026
Same author

Practical considerations when designing an online learning algorithm for an app-based mHealth intervention.

Contemporary clinical trialsยท2026
Same author

Reproducible workflow for online artificial intelligence in digital health.

Philosophical transactions. Series A, Mathematical, physical, and engineering sciencesยท2026
Same author

Personalized modeling of stress and blood pressure reactivity using mobile health data.

Npj mental health researchยท2026
Same journal

Empirical Bound Information-Directed Sampling for Norm-Agnostic Bandits.

Reinforcement learning journalยท2026
Same journal

When and Why Hyperbolic Discounting Matters for Reinforcement Learning Interventions.

Reinforcement learning journalยท2026
See all related articles

Related Experiment Video

Updated: Mar 17, 2026

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
04:35

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Published on: July 3, 2020

3.8K

Non-Stationary Latent Auto-Regressive Bandits.

Anna L Trella1, Walter Dempsey2, Asim H Gazi1

  • 1School of Engineering and Applied Sciences, Harvard University, Cambridge, MA USA.

Reinforcement Learning Journal
|March 16, 2026
PubMed
Summary
This summary is machine-generated.

Latent AR LinUCB (LARL) addresses non-stationary multi-armed bandit problems without a non-stationarity budget. This new algorithm predicts latent states to improve reward mean predictions, achieving sub-linear regret under specific conditions.

Keywords:
bandit algorithmsnon-stationarity

Related Experiment Videos

Last Updated: Mar 17, 2026

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
04:35

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Published on: July 3, 2020

3.8K

Area of Science:

  • Machine Learning
  • Reinforcement Learning
  • Artificial Intelligence

Background:

  • Traditional multi-armed bandit (MAB) algorithms often require a budget for non-stationarity, limiting their applicability in real-world scenarios.
  • Existing methods struggle when the non-stationarity mechanism is present but lacks a defined budget.
  • Modeling reward mean changes due to latent, auto-regressive (AR) states presents a significant challenge.

Purpose of the Study:

  • To develop a novel online linear contextual bandit algorithm for non-stationary MAB problems without a non-stationarity budget.
  • To address scenarios where reward means change based on an unobserved auto-regressive state.
  • To provide a method that implicitly predicts the latent state for improved reward mean estimation.

Main Methods:

  • Introduced Latent AR LinUCB (LARL), an online linear contextual bandit algorithm.
  • Reduced the non-stationary bandit problem to a linear dynamical system solvable as a linear contextual bandit.
  • LARL approximates a steady-state Kalman filter, enabling online learning of system parameters.

Main Results:

  • LARL effectively predicts reward means by implicitly learning the latent state dynamics.
  • An interpretable regret bound was derived, dependent on the environment's non-stationarity level.
  • LARL achieves sub-linear regret when the latent state process noise variance is sufficiently small relative to time steps T.

Conclusions:

  • LARL offers a viable solution for non-stationary MAB problems lacking a non-stationarity budget.
  • The algorithm demonstrates strong empirical performance, outperforming baseline methods.
  • LARL's ability to learn system parameters online and provide theoretical guarantees makes it a promising approach.