Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Decision Making: P-value Method01:09

Decision Making: P-value Method

5.8K
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim  is also stated. These statements can act as null and alternative hypotheses:  a null hypothesis would be a neutral statement while the alternative hypothesis can...
5.8K
Decision Making: Traditional Method01:14

Decision Making: Traditional Method

4.3K
The process of hypothesis testing based on the traditional method includes calculating the critical value, testing the value of the test statistic using the sample data, and interpreting these values.
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
4.3K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

115
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
115
Decision Making01:20

Decision Making

296
Decision-making is a fundamental cognitive process that involves evaluating alternatives and selecting among them. This process can range from simple choices, such as deciding what to wear, to complex decisions, like choosing a major in college or a career path. The complexity of the decision often dictates the approach we use, which can be broadly categorized into two types: automatic and controlled decision-making.
Automatic decision-making is fast, intuitive, and relies on gut feelings...
296
Expected Value01:15

Expected Value

5.3K
The expected value is known as the "long-term" average or mean. This means that over the long term of experimenting over and over, you would expect this average. The expected average is represented by the symbol μ. It is calculated as follows:
5.3K
Model Approaches for Pharmacokinetic Data: Distributed Parameter Models01:06

Model Approaches for Pharmacokinetic Data: Distributed Parameter Models

140
Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
140

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

How media competition fuels the spread of misinformation.

Science advances·2025
Same author

Neurosymbolic AI as an antithesis to scaling laws.

PNAS nexus·2025
Same author

Robustly Linearized Model Predictive Control for Nonlinear Infinite-Dimensional Systems.

IFAC-PapersOnLine·2025
Same author

Lodestar: An Integrated Embedded Real-Time Control Engine.

Control Technology and Applications. Control Technology and Applications·2024
Same author

Viability Under Degraded Control Authority.

IEEE control systems letters·2024
Same author

Class-Aware Adversarial Transformers for Medical Image Segmentation.

Advances in neural information processing systems·2023
Same journal

Classification Under Local Differential Privacy with Model Reversal and Model Averaging.

Journal of machine learning research : JMLR·2026
Same journal

Sparse Semiparametric Discriminant Analysis for High-dimensional Zero-inflated Data.

Journal of machine learning research : JMLR·2026
Same journal

Heterogeneity-aware Clustered Distributed Learning for Multi-source Data Analysis.

Journal of machine learning research : JMLR·2026
Same journal

Unsupervised Tree Boosting for Learning Probability Distributions.

Journal of machine learning research : JMLR·2026
Same journal

A Two-Stage Penalized Least Squares Method for Constructing Large Systems of Structural Equations.

Journal of machine learning research : JMLR·2026
Same journal

Bayesian Multinomial Logistic Normal Models through Marginally Latent Matrix-T Processes.

Journal of machine learning research : JMLR·2026
See all related articles

Related Experiment Video

Updated: Oct 7, 2025

An Automated T-maze Based Apparatus and Protocol for Analyzing Delay- and Effort-based Decision Making in Free Moving Rodents
07:42

An Automated T-maze Based Apparatus and Protocol for Analyzing Delay- and Effort-based Decision Making in Free Moving Rodents

Published on: August 2, 2018

13.9K

Learning and Planning for Time-Varying MDPs Using Maximum Likelihood Estimation.

Melkior Ornik1, Ufuk Topcu2

  • 1Department of Aerospace Engineering and the Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA.

Journal of Machine Learning Research : JMLR
|January 10, 2022
PubMed
Summary
This summary is machine-generated.

This study introduces a new method for agents to learn and plan in unknown, changing environments. It enables agents to adapt to dynamic conditions by accurately modeling environmental changes for improved decision-making.

Keywords:
Markov decision processeschanging environmentmaximum likelihood estimationonline learninguncertainty quantification

More Related Videos

Automated, Quantitative Cognitive/Behavioral Screening of Mice: For Genetics, Pharmacology, Animal Cognition and Undergraduate Instruction
16:23

Automated, Quantitative Cognitive/Behavioral Screening of Mice: For Genetics, Pharmacology, Animal Cognition and Undergraduate Instruction

Published on: February 26, 2014

14.5K
Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods
13:04

Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods

Published on: September 19, 2012

12.2K

Related Experiment Videos

Last Updated: Oct 7, 2025

An Automated T-maze Based Apparatus and Protocol for Analyzing Delay- and Effort-based Decision Making in Free Moving Rodents
07:42

An Automated T-maze Based Apparatus and Protocol for Analyzing Delay- and Effort-based Decision Making in Free Moving Rodents

Published on: August 2, 2018

13.9K
Automated, Quantitative Cognitive/Behavioral Screening of Mice: For Genetics, Pharmacology, Animal Cognition and Undergraduate Instruction
16:23

Automated, Quantitative Cognitive/Behavioral Screening of Mice: For Genetics, Pharmacology, Animal Cognition and Undergraduate Instruction

Published on: February 26, 2014

14.5K
Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods
13:04

Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods

Published on: September 19, 2012

12.2K

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Robotics

Background:

  • Agents often operate in environments with unpredictable changes.
  • Existing methods struggle with time-varying dynamics in Markov decision processes (MDPs).

Purpose of the Study:

  • To develop a formal approach for online learning and planning in unknown, time-varying environments.
  • To enable agents to adapt to and effectively operate within dynamic systems.

Main Methods:

  • Computing the maximally likely model of the environment based on agent observations.
  • Generalizing estimation methods for time-invariant MDPs to handle changing system dynamics.
  • Introducing uncertainty into learned time-varying models for exploration bonuses.
  • Developing a control policy balancing exploitation and exploration.

Main Results:

  • The proposed method accurately identifies system dynamics even after changes occur.
  • Generalized exploration bonuses enhance learning in dynamic environments.
  • A control policy is developed for time-varying MDPs.

Conclusions:

  • The developed approach provides a robust framework for agents in dynamic environments.
  • This method enhances adaptability and decision-making in real-world, changing conditions.
  • Demonstrated effectiveness across diverse tasks including dynamic MDPs and multi-armed bandits.