在马尔科夫采样下,为分布式DT{λ) 进行一次性平均化
Haoxing Tian1, Ioannis Ch Paschalidis2, Alex Olshevsky2
1Department of Electrical Engineering, Boston University, Boston, MA, USA.
概括
分布增强学习实现了使用TD (λ) 方法进行政策评估的线性加快. 通过独立采样和一种新的"一拍平均"技术,N代理可以更快地评估政策N倍,从而减少通信开销.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 分布式计算 (Distributed Computing) 是一种分布式计算.
背景情况:
- 强化学习 (RL) 对于顺序性决策至关重要.
- 政策评估是RL的一个基本任务.
- 分布式设置为更快的计算提供了潜力,但面临着通信挑战.
研究的目的:
- 用TD (λ) 方法研究分布式政策评估.
- 在分布式RL设置中实现线性加速度.
- 在分散的政策评估中减少沟通开销.
主要方法:
- 一个分布式设置,每个代理都有马尔科夫决策过程的副本.
- 每个代理人的独立过渡抽样.
- TD(λ) 算法用于政策评估.
- 一种新的"一次平均"程序,用于汇总代理结果.
主要成果:
- 在分布式环境中实现了TD (λ) 政策评估的线性加快.
- 证明N个代理商可以比N个代理商更快地评估政策.
- 显示当目标精度足够小时,可以实现线性加速度.
- "一拍平均"方法显著降低了通信要求.
结论:
- 分布式强化学习与独立采样和"一次性平均"使有效的政策评估成为可能.
- 通过减少通信可以实现线性加速度,优于以前的分布式方法.
- 这种方法为RL中大规模的政策评估提供了一个实用的方法.
更多相关视频
09:17Structure-Based Simulation and Sampling of Transcription Factor Protein Movements along DNA from Atomic-Scale Stepping to Coarse-Grained Diffusion
Published on: March 1, 2022
3.1K
00:10Single-Molecule Tracking Microscopy - A Tool for Determining the Diffusive States of Cytosolic Molecules
Published on: September 5, 2019
8.2K
相关概念视频
Sampling Distribution
12.3K
Given simple random samples of size n from a given population with a measured characteristic such as mean, proportion, or standard deviation for each sample, the probability distribution of all the measured characteristics is called a sampling distribution. How much the statistic varies from one sample to another is known as the sampling variability of a statistic. You typically measure the sampling variability of a statistic by its standard error. The standard error of the mean is an example...
12.3K
Cluster Sampling Method
11.8K
Appropriate sampling methods ensure that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
11.8K
Sampling Methods: Overview
288
A sample refers to a smaller subset representative of a larger population. In analytical chemistry, studying or analyzing an entire population is often impractical or impossible. Therefore, samples are used to draw inferences and generalize the whole population. The sampling method selects individuals or items from a population to create a sample. Standard sampling methods include random, judgemental, systematic, stratified, and cluster sampling.
In analytical chemistry, the choice of...
In analytical chemistry, the choice of...
288
Random Sampling Method
11.0K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest. Among the various sampling methods used by...
11.0K
Choosing Between z and t Distribution
2.8K
The z and the Student t distribution estimate the population mean using the sample mean and standard deviation. However, to decide which distribution to use for a calculation, one needs to determine the sample size, the nature of the distribution, and whether the population standard deviation is known. If the population standard deviation is known and the population is normally distributed, or if the sample size is greater than 30, the z distribution is preferred. The Student t distribution is...
2.8K
Sampling Methods: Sample Types
192
Sampling materials are classified into three main types: solid, liquid, and gas.
Solid samples include a variety of substances, such as sediments from water bodies, soil, metals, and biological tissues. Two standard methods for extracting sediments from water bodies are grab sampling and piston coring. Grab sampling involves using a device to collect a discrete sediment sample from the bottom of a water body with minimal disturbance. Grab samples do not always represent the entire area due to...
Solid samples include a variety of substances, such as sediments from water bodies, soil, metals, and biological tissues. Two standard methods for extracting sediments from water bodies are grab sampling and piston coring. Grab sampling involves using a device to collect a discrete sediment sample from the bottom of a water body with minimal disturbance. Grab samples do not always represent the entire area due to...
192
