Q-Learning Approach to Finite-Horizon H∞ Tracking With Partial Observation
IEEE Transactions on Cybernetics
|January 19, 2026
Summary
This study introduces novel model-free reinforcement learning algorithms for discrete-time systems with partial observations. These data-driven methods address finite-horizon H-infinity tracking control challenges without needing an initial policy or discount factor.
Area of Science:
- Control Theory
- Reinforcement Learning
- Game Theory
Background:
- Existing reinforcement learning (RL) methods often require full state information and are limited to infinite-horizon, time-invariant systems.
- Finite-horizon control with partial observations and unknown dynamics presents significant challenges, including the need for time-varying Riccati equations.
- Model-free approaches are desirable for systems where dynamics are unknown, relying solely on input-output data.
Purpose of the Study:
- To investigate the finite-horizon H-infinity tracking control problem for discrete-time linear systems with partial observations and unknown dynamics.
- To develop model-free reinforcement learning algorithms that overcome limitations of existing approaches, particularly regarding state information and system horizon.
- To provide a framework for solving time-varying control problems without requiring an initially admissible policy or discount factor.
Main Methods:
- Reconstruction of system state from historical input-output trajectories to create a data-driven system representation.
- Definition of a time-varying Q-function based on input-output data.
- Proposal of two minimax Q-learning algorithms designed for model-free, data-driven control.
Main Results:
- The developed algorithms successfully reconstruct system states and define input-output-based Q-functions.
- The minimax Q-learning algorithms do not require an initially admissible policy and avoid discount factors, enhancing stability guarantees.
- The framework demonstrates extensibility to both infinite-horizon and time-varying systems without structural changes.
Conclusions:
- The proposed data-driven, model-free reinforcement learning algorithms effectively address the finite-horizon H-infinity tracking control problem for discrete-time systems with partial observations.
- Theoretical convergence is proven, and simulation results validate the algorithms' effectiveness.
- This work offers a significant advancement in reinforcement learning for control, particularly for systems with unknown dynamics and partial state information.
Related Concept Videos
01:12Observational Learning
853
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
853
01:21Schwarzschild Radius and Event Horizon
2.6K
No object with a finite mass can travel faster than the speed of light in a vacuum. This fact has an interesting consequence in the domain of extremely high gravitational fields.
The minimum speed required to launch a projectile from the surface of an object to which it is gravitationally bound so that it eventually escapes the object’s gravitational field is called the escape velocity. The escape velocity is independent of the mass of the object. Merging the idea of escape...
The minimum speed required to launch a projectile from the surface of an object to which it is gravitationally bound so that it eventually escapes the object’s gravitational field is called the escape velocity. The escape velocity is independent of the mass of the object. Merging the idea of escape...
2.6K
02:30Naturalistic Observations
17.0K
If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances...
17.0K
01:16Conservation of Mass in Finite Cotrol Volume
1.8K
The principle of conservation of mass is a fundamental law in fluid mechanics and is applied using the continuity equation. We apply the concept to a finite control volume to derive the continuity equation.
A system is defined as a collection of unchanging contents, and the conservation of mass states that a system's mass is constant.
A system is defined as a collection of unchanging contents, and the conservation of mass states that a system's mass is constant.
1.8K
03:03Mixtures of Gases: Dalton's Law of Partial Pressures and Mole Fractions
43.7K
Unless individual gases chemically react with each other, the individual gases in a mixture of gases do not affect each other’s pressure. Each gas in a mixture exerts the same pressure that it would exert if it were present alone in the container. The pressure exerted by each individual gas in a mixture is called its partial pressure.
43.7K
01:14Avoidance Learning and Learned Helplessness
2.5K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
2.5K