强大而高效的部分采样算法用于大规模数据后勤回归
Jun Jin1, Shuangzhe Liu2, Tiefeng Ma3
1College of Mathematical Sciences, Yangzhou University, Yangzhou, People's Republic of China.
Journal of applied statistics
|June 12, 2024
概括
新的算法解决了大规模数据分析中的计算限制. 这些方法通过使用硬值或组合子样本来改善后勤回归估计,从而提高大数据集的效率.
科学领域:
- 统计 统计 统计 统计
- 计算机科学 计算机科学
- 数据科学数据科学数据科学
背景情况:
- 大规模数据集 (大数据) 为分析带来了计算挑战.
- 不统一的分采样减少了计算负载,但可以增加估计器变异与异质的概率.
- 现有的方法难以应对现代数据的规模和复杂性.
研究的目的:
- 在大规模数据集中开发新的算法,以改进物流回归估计.
- 解决现有的非统一部分采样方法的局限性.
- 为了提高大数据的计算效率和统计推理.
主要方法:
- 提出了一个硬值算法,用选择的值取代低次采样概率.
- 引入了组合子样本的方法,汇总来自多个子样本的估计.
- 拟议的估计器的异面性质在理论上已经确立.
主要成果:
- 硬值方法提供了一种管理亚抽样概率的方法.
- 组合子样本方法可以提高估计效率,而不需要三明治矩阵.
- 模拟和现实世界数据分析证明了这两种方法的实际有效性.
结论:
- 开发的算法为大量数据的后勤回归提供了有效的解决方案.
- 这些方法提高了计算效率和统计准确性.
- 这些发现为分析大规模复杂数据集提供了实用工具.
相关概念视频
Bootstrapping
602
The term "bootstrap" originated in the 19th century as a metaphor for self-improvement or achieving something independently, without external assistance. This concept extends to statistical bootstrapping, a self-contained method for estimating population parameters through resampling, even though it can be computationally intensive. Developed by the American statistician Dr. Bradley Efron in 1979, bootstrapping provides a robust way to perform inference when the original sample size is...
602
Sampling Methods: Sample Types
214
Sampling materials are classified into three main types: solid, liquid, and gas.
Solid samples include a variety of substances, such as sediments from water bodies, soil, metals, and biological tissues. Two standard methods for extracting sediments from water bodies are grab sampling and piston coring. Grab sampling involves using a device to collect a discrete sediment sample from the bottom of a water body with minimal disturbance. Grab samples do not always represent the entire area due to...
Solid samples include a variety of substances, such as sediments from water bodies, soil, metals, and biological tissues. Two standard methods for extracting sediments from water bodies are grab sampling and piston coring. Grab sampling involves using a device to collect a discrete sediment sample from the bottom of a water body with minimal disturbance. Grab samples do not always represent the entire area due to...
214
Random Sampling Method
11.0K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest. Among the various sampling methods used by...
11.0K
Sampling Methods: Overview
309
A sample refers to a smaller subset representative of a larger population. In analytical chemistry, studying or analyzing an entire population is often impractical or impossible. Therefore, samples are used to draw inferences and generalize the whole population. The sampling method selects individuals or items from a population to create a sample. Standard sampling methods include random, judgemental, systematic, stratified, and cluster sampling.
In analytical chemistry, the choice of...
In analytical chemistry, the choice of...
309
Sampling Plans
180
Sampling is a crucial step in analytical chemistry, allowing researchers to collect representative data from a large population. Common sampling methods include random, judgmental, systematic, stratified, and cluster sampling.
Random sampling is a method where each member of the population has an equal chance of being selected for the sample. It involves selecting individuals randomly, often using random number generators or lottery-type methods. For example, when analyzing the properties of a...
Random sampling is a method where each member of the population has an equal chance of being selected for the sample. It involves selecting individuals randomly, often using random number generators or lottery-type methods. For example, when analyzing the properties of a...
180
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
48
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
48


