Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Decision Making: P-value Method01:09

Decision Making: P-value Method

The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim  is also stated. These statements can act as null and alternative hypotheses:  a null hypothesis would be a neutral statement while the alternative hypothesis can have a...
Decision Making01:20

Decision Making

Decision-making is a fundamental cognitive process that involves evaluating alternatives and selecting among them. This process can range from simple choices, such as deciding what to wear, to complex decisions, like choosing a major in college or a career path. The complexity of the decision often dictates the approach we use, which can be broadly categorized into two types: automatic and controlled decision-making.
Automatic decision-making is fast, intuitive, and relies on gut feelings...
Decision Making: Traditional Method01:14

Decision Making: Traditional Method

The process of hypothesis testing based on the traditional method includes calculating the critical value, testing the value of the test statistic using the sample data, and interpreting these values.
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
Mechanistic Models: Compartment Models in Individual and Population Analysis01:23

Mechanistic Models: Compartment Models in Individual and Population Analysis

Mechanistic models are utilized in individual analysis using single-source data, but imperfections arise due to data collection errors, preventing perfect prediction of observed data. The mathematical equation involves known values (Xi), observed concentrations (Ci), measurement errors (εi), model parameters (ϕj), and the related function (ƒi) for i number of values. Different least-squares metrics quantify differences between predicted and observed values. The ordinary least squares (OLS)...
Introduction to Partial Derivatives01:25

Introduction to Partial Derivatives

In many real-world situations, an output depends on more than one input. In a high-tech assembly plant, total production may depend on technician labor and machine capacity at the same time. This relationship can be represented by a continuous function P(T, M), where T denotes technician labor input, and M denotes machine capacity. When demand increases, but the budget remains fixed, the manager must determine which input will improve production more efficiently.Partial derivatives provide a...
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Emerging Progress of walnut protein: A critical review on structural characterization, functional optimization, and application expansion.

Food chemistry·2026
Same author

High-throughput phenotyping for climate-resilient forests: integrating multi-sensor fusion and root-shoot dynamics.

Frontiers in plant science·2026
Same author

Soil-Nutrient Depletion and Microbial Community Restructuring in Continuous Celery Cropping: Opposing Responses of Bacteria and Fungi.

Biology·2026
Same author

COST1 stimulates RHD3 GTPase activity to maintain ER morphology and plant growth in <i>Arabidopsis</i>.

Science advances·2026
Same author

From waste to wealth: AI-driven enzyme innovation for straw transformation.

Bioresource technology·2026
Same author

Macrophage-MSCs Interplay in Orthodontic Bone Remodelling: Mechanics-Immunity-Stem Cell Axis.

International dental journal·2026
Same journal

Strategic Ability Updating in Concurrent Games by Coalitional Commitment.

IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society·2015
Same journal

Meta-Analysis of the First Facial Expression Recognition Challenge.

IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society·2012
Same journal

Adjustable model-based fusion method for multispectral and panchromatic images.

IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society·2012
Same journal

Face Feature Weighted Fusion Based on Fuzzy Membership Degree for Video Face Recognition.

IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society·2012
Same journal

A New Adaptive Fast Cellular Automaton Neighborhood Detection and Rule Identification Algorithm.

IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society·2012
Same journal

Human-arm-and-hand-dynamic model with variability analyses for a stylus-based haptic interface.

IEEE transactions on systems, man, and cybernetics. Part B, Cybernetics : a publication of the IEEE Systems, Man, and Cybernetics Society·2012
See all related articles

Related Experiment Video

Updated: Jun 27, 2026

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents
07:05

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents

Published on: September 10, 2018

Partially observable Markov decision processes and performance sensitivity analysis.

Yanjie Li1, Baoqun Yin, Hongsheng Xi

  • 1Department of Automation, University of Science and Technology of China, Hefei 230026, China. whylyj@ustc.edu

IEEE Transactions on Systems, Man, and Cybernetics. Part B, Cybernetics : a Publication of the IEEE Systems, Man, and Cybernetics Society
|November 22, 2008
PubMed
Summary
This summary is machine-generated.

This study extends sensitivity-based optimization to partially observable Markov decision processes (POMDPs). New methods estimate performance gradients and derive optimality conditions for average reward POMDPs.

More Related Videos

Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments
13:00

Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments

Published on: January 23, 2017

Related Experiment Videos

Last Updated: Jun 27, 2026

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents
07:05

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents

Published on: September 10, 2018

Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments
13:00

Measuring Attention and Visual Processing Speed by Model-based Analysis of Temporal-order Judgments

Published on: January 23, 2017

Area of Science:

  • Artificial Intelligence
  • Control Theory
  • Operations Research

Background:

  • Sensitivity-based optimization is crucial for Markov systems.
  • Policy-iteration and gradient estimation are established for Markov decision processes (MDPs).

Purpose of the Study:

  • Extend sensitivity-based optimization to average reward partially observable Markov decision processes (POMDPs).
  • Develop methods for performance gradient estimation and derive optimality conditions for POMDPs.

Main Methods:

  • Derivation of performance-difference and performance-derivative formulas for POMDPs.
  • Development of a novel method for estimating performance gradients using the performance-derivative formula.
  • Formulation of a sufficient optimality condition based on the performance-difference formula, avoiding discounted rewards.

Main Results:

  • Established performance-difference and performance-derivative formulas for POMDPs.
  • Introduced a new gradient estimation technique for POMDPs.
  • Derived a sufficient optimality condition for average reward POMDPs.

Conclusions:

  • The proposed methods enable sensitivity-based optimization for average reward POMDPs.
  • A policy-iteration algorithm is presented to find near-optimal finite-state-controller policies for POMDPs.