Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Outliers and Influential Points01:08

Outliers and Influential Points

4.2K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
4.2K
What Are Outliers?01:12

What Are Outliers?

4.0K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
4.0K
Quantifying and Rejecting Outliers: The Grubbs Test01:02

Quantifying and Rejecting Outliers: The Grubbs Test

1.8K
Sometimes, a data set can have a recorded numerical observation that greatly  deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier.  To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.8K
Detection of Gross Error: The Q Test01:00

Detection of Gross Error: The Q Test

6.3K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.3K
Modified Boxplots00:57

Modified Boxplots

9.9K
A standard box and whisker plot informs us about the spread of the data in a given sample. One can identify the minimum value, maximum value, first quartile value, second quartile or median value, and third quartile.
However, the box plot does not tell the reader about outliers - values that lie far from the center of the data. We can modify the standard box and whisker plot to identify the outliers and visualize the actual spread of the data in a sample.
Initially, we calculate the adjusted...
9.9K
Unusual Results01:16

Unusual Results

3.3K
Unusual results are those that have a very low chance of occurring. Unusual results can be identified using probabilities and the range rule of thumb. In problems involving probability, unusual results can be observed in 2 instances – an unusually high number of successes or an unusually low number of successes.
According to the range rule of thumb, any value above or below two standard deviations, 2σ  from the mean, μ  is considered unusual.
Maximum unusual value =...
3.3K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Online Monitoring and Fault Diagnosis for High-Dimensional Stream with Application in Electron Probe X-Ray Microanalysis.

Entropy (Basel, Switzerland)·2025
Same author

Modelling and monitoring social network change based on exponential random graph models.

Journal of applied statistics·2024
Same author

[Prevalence and prognostic factors for postoperative complications of uvulopalatopharyngoplasty in patients with obstructive sleep apnea hypopnea syndrome].

Lin chuang er bi yan hou tou jing wai ke za zhi = Journal of clinical otorhinolaryngology head and neck surgery·2008
Same author

[Transurethral electrotomy for cystis vesicular seminalis induced by obstruction of the distal end of the ejaculatory duct].

Zhonghua nan ke xue = National journal of andrology·2008
Same author

[Effects of testosterone on the proliferation of rat corpus cavernosum cells in vitro].

Zhonghua nan ke xue = National journal of andrology·2008
Same author

Identification of 4-aminopyrazolylpyrimidines as potent inhibitors of Trk kinases.

Journal of medicinal chemistry·2008

Related Experiment Video

Updated: Aug 12, 2025

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

2.6K

Identification of outlying observations for large-dimensional data.

Tao Wang1, Xiaona Yang2, Yunfei Guo3,4

  • 1School of Mathematics and Statistics, Huaiyin Normal University, Huaian City, People's Republic of China.

Journal of Applied Statistics
|January 26, 2023
PubMed
Summary

This study introduces a novel two-stage method for detecting outliers in large datasets. The procedure effectively identifies unusual data points, offering improved accuracy and reliability for data analysis.

Keywords:
Outlier identificationasymptotic distributionlarge-dimension statisticsmultiple hypothesis testingrobust statistics

More Related Videos

Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data
14:27

Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data

Published on: June 26, 2013

15.7K
Large-scale Reconstructions and Independent, Unbiased Clustering Based on Morphological Metrics to Classify Neurons in Selective Populations
12:27

Large-scale Reconstructions and Independent, Unbiased Clustering Based on Morphological Metrics to Classify Neurons in Selective Populations

Published on: February 15, 2017

7.0K

Related Experiment Videos

Last Updated: Aug 12, 2025

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

2.6K
Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data
14:27

Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data

Published on: June 26, 2013

15.7K
Large-scale Reconstructions and Independent, Unbiased Clustering Based on Morphological Metrics to Classify Neurons in Selective Populations
12:27

Large-scale Reconstructions and Independent, Unbiased Clustering Based on Morphological Metrics to Classify Neurons in Selective Populations

Published on: February 15, 2017

7.0K

Area of Science:

  • Statistics
  • Data Mining
  • Machine Learning

Background:

  • Identifying outlying observations is crucial for robust data analysis.
  • Existing methods may struggle with large-dimensional datasets.

Purpose of the Study:

  • To propose an effective two-stage procedure for outlier identification in large-dimensional data.
  • To develop a refined algorithm for enhanced outlier detection performance.

Main Methods:

  • A two-stage procedure utilizing a max-normal statistic and a clean subset.
  • Exploration of asymptotic distribution for threshold determination.
  • Development of a one-step refined algorithm to improve identification power.

Main Results:

  • The proposed method demonstrates significant advantages in outlier identification.
  • Effective control over misjudgment rates was achieved.
  • Validated through simulations and real-world data analysis.

Conclusions:

  • The novel two-stage procedure offers a powerful and reliable approach to outlier detection.
  • The refined algorithm enhances the practical applicability for large datasets.