Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Outliers and Influential Points

Outliers and Influential Points

An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the vertical...

What Are Outliers?

What Are Outliers?

Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...

Quantifying and Rejecting Outliers: The Grubbs Test

Quantifying and Rejecting Outliers: The Grubbs Test

Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This number is...

Detection of Gross Error: The Q Test

Detection of Gross Error: The Q Test

When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...

Modified Boxplots

Modified Boxplots

A standard box and whisker plot informs us about the spread of the data in a given sample. One can identify the minimum value, maximum value, first quartile value, second quartile or median value, and third quartile.
However, the box plot does not tell the reader about outliers - values that lie far from the center of the data. We can modify the standard box and whisker plot to identify the outliers and visualize the actual spread of the data in a sample.
Initially, we calculate the adjusted...

Trimmed Mean

Trimmed Mean

While measuring the mean of a data set, care needs to be taken when associating the mean to its central tendency. The same goes for the arithmetic mean, the geometric mean, or the harmonic mean. This is because the presence of a single outlier data value can significantly affect the mean. That is, the mean is sensitive to fluctuations in the data set.
Although certain measures of central tendency are not sensitive to outliers, there are alternative versions of the mean that get around the...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Special collection devoted to the VIII "metabolomics circle" conference organized by the Polish metabolomics society.

Metabolomics : Official journal of the Metabolomic Society·2024

Same author

Common and distinct pollution sources identified from ambient PM<sub>2.5</sub> concentrations in two sites of Los Angeles Basin from 2005 to 2019.

Environmental pollution (Barking, Essex : 1987)·2023

Same author

Monitoring the concentrations of Cd, Cu, Pb, Ni, Cr, Zn, Mn and Fe in cultivated Haplic Luvisol soils using near-infrared reflectance spectroscopy and chemometrics.

Talanta·2022

Same author

Multi-wavelength imaging of HPTLC plates using a constructed illumination chamber with a smartphone camera as the detector.

Talanta·2020

Same author

Detecting chemical markers to uncover counterfeit rebated excise duty diesel oil.

Talanta·2019

Same author

Identification of ground meat species using near-infrared spectroscopy and class modeling techniques - Aspects of optimization and validation using a one-class classification model.

Meat science·2018

Search research articles

Related Experiment Video

Updated: Jun 27, 2026

An Introduction to Processing, Fitting, and Interpreting Transient Absorption Data

An Introduction to Processing, Fitting, and Interpreting Transient Absorption Data

Published on: February 16, 2024

Dealing with missing values and outliers in principal component analysis.

I Stanimirova¹, M Daszykowski, B Walczak

¹Department of Chemometrics, Institute of Chemistry, The University of Silesia, 9 Szkolna Street, 40-006 Katowice, Poland.

|December 17, 2008

Summary

This summary is machine-generated.

This study introduces a robust method for handling missing data and outliers in Principal Component Analysis (PCA). The new approach effectively processes contaminated datasets with missing values, improving data analysis reliability.

More Related Videos

Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data

Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data

Published on: June 26, 2013

Basics of Multivariate Analysis in Neuroimaging Data

Basics of Multivariate Analysis in Neuroimaging Data

Published on: July 24, 2010

Related Experiment Videos

Last Updated: Jun 27, 2026

An Introduction to Processing, Fitting, and Interpreting Transient Absorption Data

An Introduction to Processing, Fitting, and Interpreting Transient Absorption Data

Published on: February 16, 2024

Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data

Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data

Published on: June 26, 2013

Basics of Multivariate Analysis in Neuroimaging Data

Basics of Multivariate Analysis in Neuroimaging Data

Published on: July 24, 2010

Area of Science:

Data Science
Statistical Analysis
Environmental Science

Background:

Missing values and outliers pose significant challenges in Principal Component Analysis (PCA).
Existing methods often struggle to address both issues simultaneously, especially in contaminated datasets.

Purpose of the Study:

To propose an efficient methodology for simultaneously handling missing values and outlying observations in PCA.
To develop a robust approach for Principal Component Analysis (PCA) that is resilient to data contamination.

Main Methods:

Utilizing a robust technique to derive robust principal components.
Integrating the expectation-maximization (EM) algorithm to effectively process datasets with missing elements.

Main Results:

The proposed strategy demonstrates strong performance even with highly contaminated data.
The methodology successfully handles varying amounts of missing data elements.

Conclusions:

The combined robust PCA and EM approach provides an effective solution for analyzing datasets with missing values and outliers.
This method enhances the reliability of Principal Component Analysis (PCA) in real-world applications, such as environmental data analysis.