Related Experiment Video
Updated: Feb 24, 2026
13:04
Measuring the Subjective Value of Risky and Ambiguous Options using Experimental Economics and Functional MRI Methods
Published on: September 19, 2012
12.5K
Optimism in the face of uncertainty supported by a statistically-designed multi-armed bandit algorithm
1Graduate School of Science and Engineering, Tokyo Denki University, Japan; School of Science and Engineering, Tokyo Denki University, Japan.
Bio Systems
|August 26, 2017
Summary
The unified Overtaking method enhances multi-armed bandit problem-solving by redefining value functions. This new approach, based on statistical confidence intervals, outperforms the UCB algorithm for exponentially distributed rewards.
Area of Science:
- Decision Sciences
- Machine Learning
- Optimization Theory
Background:
- Sequential decision-making problems often involve uncertainty.
- The Overtaking method is an effective algorithm for multi-armed bandit problems, previously defined by heuristic patterns.
- Optimism in the face of uncertainty is a key principle in these problems.
Purpose of the Study:
- To redefine and unify the value functions of the Overtaking method.
- To enhance the universality and applicability of the Overtaking method.
- To propose a statistics-based interpretation of optimism in sequential decision-making.
Main Methods:
- Unified formulation of the Overtaking method using upper confidence bounds on expected rewards.
- Development of the Overtaking method for exponentially distributed rewards.
- Numerical analysis and comparison with the UCB algorithm.
Main Results:
- The unified Overtaking method demonstrates enhanced universality.
- A novel Overtaking method for exponentially distributed rewards was developed and analyzed.
- The proposed method statistically outperforms the UCB algorithm on average.
Conclusions:
- The principle of optimism in sequential decision-making can be viewed as a statistical consequence of the law of large numbers.
- Unified formulations improve algorithm universality in multi-armed bandit problems.
- The new Overtaking method offers improved performance for specific reward distributions.
Related Concept Videos
01:30Unrealistic Optimism Bias
269
Unrealistic optimism bias is the tendency to overestimate the likelihood of positive outcomes. This cognitive bias makes individuals believe they are less likely to experience failures, setbacks, or risks and more likely to succeed than others. For example, people may assume they are less prone to health issues, accidents, or financial struggles than their peers, even when they share similar risk factors.One key component of this bias is the above-average effect, where individuals perceive...
269
00:54Uncertainty: Confidence Intervals
11.8K
The confidence interval is the range of values around the mean that contains the true mean. It is expressed as a probability percentage. The interpretation of a 95% confidence interval, for instance, is that the statistician is 95% confident that the true mean falls within the interval. The upper and lower limits of this range are known as confidence limits. The confidence limits for the true mean are estimated from the sample's mean, the standard deviation, and the statistical factor...
11.8K
00:59Propagation of Uncertainty from Random Error
2.0K
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
2.0K
00:59Uncertainty: Overview
1.8K
In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
1.8K
01:10Propagation of Uncertainty from Systematic Error
1.5K
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
1.5K
01:13Randomized Experiments
9.1K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
9.1K