反事实增强重要性抽样用于半线下政策评估
1Computer Science & Engineering, University of Michigan, Ann Arbor, MI, USA.
Advances in neural information processing systems
|December 15, 2025
概括
本研究介绍了一种半线下评估框架,用于高风险领域的强化学习 (RL). 它使用人类注释来改善政策评估,克服纯粹离线或不安全的在线方法的局限性.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算统计学 计算统计学
背景情况:
- 使用观测数据的政策之外的评估 (OPE) 受到分配转移的限制.
- 在高风险领域,由于安全问题,在线评估往往是不可行的.
研究的目的:
- 为强化学习 (RL) 提出一个半线下评估框架.
- 将对反事实轨迹的人类注释纳入,以改进OPE.
- 开发新的OPE估计器,以减轻偏差和差异.
主要方法:
- 开发了一种半线下评估框架,将线下数据与人类注释相结合.
- 设计了一个新的OPE估计器家族,使用重要性抽样 (IS) 和一种新的权重方案.
- 分析理论性质并进行概念验证实验.
主要成果:
- 拟议的方法包括反事实注释,而不会引入偏见.
- 该方法显示了与标准IS估计器相比,减少偏差和差异的潜力.
- 实验显示出高于纯线下IS估计器的性能,即使有不完美的注释.
结论:
- 半线下框架使在关键应用中实现更安全,更可靠的RL政策评估.
- 以人为中心的注释设计对于有效实施至关重要.
- 这项工作通过解决评估挑战,促进RL在高风险领域的采用.
相关概念视频
Decision Making: P-value Method
6.8K
The process of hypothesis testing based on the P-value method includes calculating the P- value using the sample data and interpreting it.
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
6.8K
Randomized Experiments
8.8K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
8.8K
Sampling Plans
861
Sampling is a crucial step in analytical chemistry, allowing researchers to collect representative data from a large population. Common sampling methods include random, judgmental, systematic, stratified, and cluster sampling.
Random sampling is a method where each member of the population has an equal chance of being selected for the sample. It involves selecting individuals randomly, often using random number generators or lottery-type methods. For example, when analyzing the properties of a...
Random sampling is a method where each member of the population has an equal chance of being selected for the sample. It involves selecting individuals randomly, often using random number generators or lottery-type methods. For example, when analyzing the properties of a...
861
Convenience Sampling Method
10.8K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population.
Convenience sampling is a non-random method of sample selection; this method selects individuals that are easily accessible and may result in biased data. For example, a marketing...
Convenience sampling is a non-random method of sample selection; this method selects individuals that are easily accessible and may result in biased data. For example, a marketing...
10.8K
Random Sampling Method
14.0K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest. Among the various sampling methods used by...
14.0K
Assumptions of Survival Analysis
382
Survival models analyze the time until one or more events occur, such as death in biological organisms or failure in mechanical systems. These models are widely used across fields like medicine, biology, engineering, and public health to study time-to-event phenomena. To ensure accurate results, survival analysis relies on key assumptions and careful study design.
382

