Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Distributions to Estimate Population Parameter01:26

Distributions to Estimate Population Parameter

4.8K
The accurate values of population parameters such as population proportion, population mean, and population standard deviation (or variance) are usually unknown. These are fixed values that can only be estimated from the data collected from the samples. The estimates of each of these parameters are sample proportion, the sample mean, and sample standard deviation (or variance). To obtain the values of these sample statistics, data are required that have particular distribution and central...
4.8K
Choosing Between z and t Distribution01:25

Choosing Between z and t Distribution

3.4K
The z and the Student t distribution estimate the population mean using the sample mean and standard deviation. However, to decide which distribution to use for a calculation, one needs to determine the sample size, the nature of the distribution, and whether the population standard deviation is known. If the population standard deviation is known and the population is normally distributed, or if the sample size is greater than 30, the z distribution is preferred. The Student t distribution is...
3.4K
Central Limit Theorem01:14

Central Limit Theorem

18.7K
The central limit theorem, abbreviated as clt, is one of the most powerful and useful ideas in all of statistics. The central limit theorem for sample means says that if you repeatedly draw samples of a given size and calculate their means, and create a histogram of those means, then the resulting histogram will tend to have an approximate normal bell shape. In other words, as sample sizes increase, the distribution of means follows the normal distribution more closely.
The sample size, n, that...
18.7K
Quantifying and Rejecting Outliers: The Grubbs Test01:02

Quantifying and Rejecting Outliers: The Grubbs Test

3.1K
Sometimes, a data set can have a recorded numerical observation that greatly  deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier.  To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
3.1K
Sampling Distribution01:12

Sampling Distribution

16.1K
Given simple random samples of size n from a given population with a measured characteristic such as mean, proportion, or standard deviation for each sample, the probability distribution of all the measured characteristics is called a sampling distribution. How much the statistic varies from one sample to another is known as the sampling variability of a statistic. You typically measure the sampling variability of a statistic by its standard error. The standard error of the mean is an example...
16.1K
Estimating Population Mean with Unknown Standard Deviation01:22

Estimating Population Mean with Unknown Standard Deviation

8.5K
In practice, we rarely know the population standard deviation. In the past, when the sample size was large, this did not present a problem to statisticians. They used the sample standard deviation s as an estimate for σ and proceeded as before to calculate a confidence interval with close enough results. However, statisticians ran into problems when the sample size was small. A small sample size caused inaccuracies in the confidence interval.
William S. Gosset (1876–1937) of the...
8.5K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Artificial intelligence deep learning algorithm for discriminating ungradable optical coherence tomography three-dimensional volumetric optic disc scans.

Neurophotonicsยท2019
Same author

LncTarD: a manually-curated database of experimentally-supported functional lncRNA-target regulations in human diseases.

Nucleic acids researchยท2019
Same author

Comparative Proteomics Analysis of Human Liver Microsomes and S9 Fractions.

Drug metabolism and disposition: the biological fate of chemicalsยท2019
Same author

miR-181b/Oncostatin m axis inhibits prostate cancer bone metastasis via modulating osteoclast differentiation.

Journal of cellular biochemistryยท2019
Same author

Transcriptomic and metabolomic profiling provide novel insights into fruit development and flesh coloration in Prunus mira Koehne, a special wild peach species.

BMC plant biologyยท2019
Same author

Genome-wide Dissection of Co-selected UV-B Responsive Pathways in the UV-B Adaptation of Qingke.

Molecular plantยท2019
Same journal

Modeling the impact of budget limitation on the screening and treatment pathway of HPV-induced precancerous cervical lesions.

Mathematical biosciences and engineering : MBEยท2026
Same journal

Modeling the effects of trait-mediated dispersal on coexistence of two species: Competition and non-consumptive predator-prey.

Mathematical biosciences and engineering : MBEยท2026
Same journal

A close look at the viral reduction rate in target cell limited models.

Mathematical biosciences and engineering : MBEยท2026
Same journal

A stochastic agent-based model for simulating tumor-immune dynamics and evaluating therapeutic strategies.

Mathematical biosciences and engineering : MBEยท2026
Same journal

Addressing domain shift via imbalance-aware domain adaptation in embryo development assessment.

Mathematical biosciences and engineering : MBEยท2026
Same journal

Effect of drug resistance on an HIV epidemic in heterogeneous populations.

Mathematical biosciences and engineering : MBEยท2026
See all related articles

Related Experiment Video

Updated: Nov 19, 2025

Modeling the Size Spectrum for Macroinvertebrates and Fishes in Stream Ecosystems
07:41

Modeling the Size Spectrum for Macroinvertebrates and Fishes in Stream Ecosystems

Published on: July 30, 2019

7.8K

A distributed quantile estimation algorithm of heavy-tailed distribution with massive datasets.

Xiaoyue Xie1,2, Jian Shi1,2

  • 1Academy of Mathematics and Systems Science, Chinese Academy of Science, Beijing 100190, China.

Mathematical Biosciences and Engineering : MBE
|February 2, 2021
PubMed
Summary
This summary is machine-generated.

Estimating high quantiles from massive datasets is challenging. This study introduces a distributed algorithm using the alternating direction method of multipliers for accurate quantile estimation in heavy-tailed distributions.

Keywords:
Peak Over Threshold methodbig datadistributed algorithmheavy-tailed distributionhigh quantile estimation

Related Experiment Videos

Last Updated: Nov 19, 2025

Modeling the Size Spectrum for Macroinvertebrates and Fishes in Stream Ecosystems
07:41

Modeling the Size Spectrum for Macroinvertebrates and Fishes in Stream Ecosystems

Published on: July 30, 2019

7.8K

Area of Science:

  • Statistics
  • Data Science
  • Computational Statistics

Background:

  • Quantile estimation in massive datasets presents significant statistical challenges.
  • Heavy-tailed distributions require specialized methods for accurate quantile estimation.
  • Existing methods may struggle with the scale and complexity of big data.

Purpose of the Study:

  • To develop a distributed algorithm for estimating high quantiles of heavy-tailed distributions using massive datasets.
  • To address the computational and statistical challenges of big data quantile estimation.
  • To provide a feasible and efficient method for extreme quantile computation.

Main Methods:

  • A distributed algorithm employing the alternating direction method of multipliers (ADMM) for parameter estimation.
  • Utilizing the Generalized Pareto Distribution (GPD) for modeling heavy-tailed data.
  • Parameter estimation via the Peak Over Threshold (POT) method within a distributed framework.

Main Results:

  • The proposed distributed algorithm is proven to converge to a stationary solution under specific step-size conditions.
  • Numerical studies and real-world data analysis demonstrate the algorithm's feasibility and efficiency.
  • The method effectively estimates high quantiles for heavy-tailed distributions in massive datasets.

Conclusions:

  • The developed distributed algorithm offers a robust solution for high quantile estimation with big data.
  • The approach is efficient and accurate, particularly for heavy-tailed distributions.
  • This method provides a reliable tool for analyzing extreme values in large datasets.