Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Decision Making: P-value Method01:09

Decision Making: P-value Method

5.6K
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim  is also stated. These statements can act as null and alternative hypotheses:  a null hypothesis would be a neutral statement while the alternative hypothesis can...
5.6K
Percentile01:18

Percentile

7.0K
A percentile indicates the relative standing of a data value when data are sorted into numerical order from smallest to largest. It represents the percentages of data values that are less than or equal to the pth percentile. For example, 15% of data values are less than or equal to the 15th percentile.
7.0K
Decision Making: Traditional Method01:14

Decision Making: Traditional Method

4.1K
The process of hypothesis testing based on the traditional method includes calculating the critical value, testing the value of the test statistic using the sample data, and interpreting these values.
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
4.1K
Decision Making01:20

Decision Making

189
Decision-making is a fundamental cognitive process that involves evaluating alternatives and selecting among them. This process can range from simple choices, such as deciding what to wear, to complex decisions, like choosing a major in college or a career path. The complexity of the decision often dictates the approach we use, which can be broadly categorized into two types: automatic and controlled decision-making.
Automatic decision-making is fast, intuitive, and relies on gut feelings...
189
Probability Distributions01:32

Probability Distributions

7.8K
 The probability of a random variable x  is the likelihood of its occurrence. A probability distribution represents the probabilities of a random variable using a formula, graph, or table. There are two types of probability distribution– discrete probability distribution and continuous probability distribution.
A discrete probability distribution is a probability distribution of discrete random variables. It can be categorized into binomial probability distribution and Poisson...
7.8K
Quartile01:15

Quartile

4.5K
Quartiles are numbers that separate the data into quarters. Quartiles may or may not be part of the data. To find the quartiles, first, find the median or second quartile. The first quartile, Q1, is the middle value of the lower half of the data, and the third quartile, Q3, is the middle value, or median, of the upper half of the data. To get the idea, consider the same data set:
1; 1; 2; 2; 4; 6; 6.8; 7.2; 8; 8.3; 9; 10; 10; 11.5
The median or second quartile is seven. The lower half of the...
4.5K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Novel susceptibility genes for varicose veins revealed by a cross-tissue transcriptome-wide association study.

Science progress·2026
Same author

Bidirectional associations between mental health conditions and cognitive impairment in patients with pain conditions of the back, neck, and spine: A population-based study.

PloS one·2026
Same author

Biodegradable Zn-Based Implants: Progress, Challenges, and Pathways toward Clinical Translation.

Advanced science (Weinheim, Baden-Wurttemberg, Germany)·2026
Same author

Exploring Acylcarnitine Metabolism Using Reverse Metabolomics.

Analytical chemistry·2026
Same author

Enhancing randomized controlled trials through smartwatch-guided participant matching for infectious disease outcomes.

Scientific reports·2026
Same author

Emulsion Gel for Intestine-Specific Enzyme-Triggered Release of Probiotics.

Small (Weinheim an der Bergstrasse, Germany)·2026
Same journal

Dynamics of Drug Resistance: Optimal Control of an Infectious Disease.

Operations research·2021
Same journal

Inverse Optimization: A New Perspective on the Black-Litterman Model.

Operations research·2014
Same journal

Optimal Breast Biopsy Decision-Making Based on Mammographic Features and Demographic Factors.

Operations research·2011
Same journal

Controlling Co-Epidemics: Analysis of HIV and Tuberculosis Infection Dynamics.

Operations research·2009
Same journal

A model for making project funding decisions at the National Cancer Institute.

Operations research·1992
Same journal

Bounds on a trauma outcome function via optimization.

Operations research·1991
See all related articles

Related Experiment Video

Updated: Aug 30, 2025

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents
07:05

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents

Published on: September 10, 2018

6.1K

Quantile Markov Decision Processes.

Xiaocheng Li1, Huaiyang Zhong1, Margaret L Brandeau1

  • 1Department of Management Science and Engineering, Stanford University, Stanford, CA, 94305.

Operations Research
|August 29, 2022
PubMed
Summary
This summary is machine-generated.

This study introduces quantile Markov decision processes (QMDPs) to optimize reward quantiles, not just expectations. A dynamic programming algorithm is presented for optimal policies, applicable to risk-averse decision-making.

Keywords:
Dynamic ProgrammingMarkov Decision ProcessMedical Decision MakingQuantileRisk Measure

More Related Videos

Measuring Delay Discounting in Humans Using an Adjusting Amount Task
07:47

Measuring Delay Discounting in Humans Using an Adjusting Amount Task

Published on: January 9, 2016

15.5K
Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods
13:04

Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods

Published on: September 19, 2012

12.2K

Related Experiment Videos

Last Updated: Aug 30, 2025

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents
07:05

Operant Protocols for Assessing the Cost-benefit Analysis During Reinforced Decision Making by Rodents

Published on: September 10, 2018

6.1K
Measuring Delay Discounting in Humans Using an Adjusting Amount Task
07:47

Measuring Delay Discounting in Humans Using an Adjusting Amount Task

Published on: January 9, 2016

15.5K
Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods
13:04

Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods

Published on: September 19, 2012

12.2K

Area of Science:

  • Operations Research
  • Decision Theory
  • Reinforcement Learning

Background:

  • Traditional Markov decision processes (MDPs) focus on maximizing expected cumulative rewards.
  • Many real-world scenarios require optimizing specific reward quantiles for risk-averse decision-making.
  • Existing MDP frameworks may not adequately address quantile optimization objectives.

Purpose of the Study:

  • To introduce and define the quantile Markov decision process (QMDP) framework.
  • To develop analytical results for the optimal QMDP value function.
  • To present a dynamic programming algorithm for solving QMDPs and related risk-sensitive objectives.

Main Methods:

  • Development of analytical characterizations for the optimal QMDP value function.
  • Design of a dynamic programming-based algorithm for policy optimization.
  • Extension of the algorithm to handle Conditional Value-at-Risk (CVaR) objectives in MDPs.

Main Results:

  • The paper provides theoretical insights into optimizing reward quantiles within MDPs.
  • An efficient dynamic programming algorithm is proposed for finding optimal QMDP policies.
  • The algorithm's applicability is demonstrated for CVaR objectives.

Conclusions:

  • The QMDP framework offers a powerful approach for decision-making under quantile-based objectives.
  • The presented dynamic programming algorithm effectively solves QMDPs and related risk-sensitive problems.
  • The model has practical implications, as shown in an HIV treatment initiation case study.