Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Quantifying and Rejecting Outliers: The Grubbs Test01:02

Quantifying and Rejecting Outliers: The Grubbs Test

1.6K
Sometimes, a data set can have a recorded numerical observation that greatly  deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier.  To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.6K
Outliers and Influential Points01:08

Outliers and Influential Points

4.1K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
4.1K
Unusual Results01:16

Unusual Results

3.2K
Unusual results are those that have a very low chance of occurring. Unusual results can be identified using probabilities and the range rule of thumb. In problems involving probability, unusual results can be observed in 2 instances – an unusually high number of successes or an unusually low number of successes.
According to the range rule of thumb, any value above or below two standard deviations, 2σ  from the mean, μ  is considered unusual.
Maximum unusual value =...
3.2K
Relative Frequency Histogram01:14

Relative Frequency Histogram

5.5K
The relative frequency depicts the proportion of data points that have each value. The frequency tells the number of data points that have each value. Like the histogram, a relative frequency histogram also has the same shape with a horizontal scale (the x-axis), but the vertical scale (the y-axis) is marked with relative frequencies (percentages of the whole) instead of actual frequencies. A relative frequency histogram is a graphical representation of a frequency distribution where the...
5.5K
Statistical Analysis: Overview01:11

Statistical Analysis: Overview

6.6K
When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
6.6K
Probability Histograms01:17

Probability Histograms

11.6K
A probability histogram is a visual representation of a probability distribution. Similar a typical histogram, the probability histogram consists of contiguous (adjoining) boxes. It has both a horizontal axis and a vertical axis. The horizontal axis is labeled with what the data represents. The vertical axis is labeled with probability. Each rectangular bar in the histogram is 1 unit wide, which suggests that the area under each bar equals the probability, P(x), where x is 1, 2, 3, and so on.
11.6K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

To Impute or Not To Impute in Untargeted Metabolomics─That is the Compositional Question.

Journal of the American Society for Mass Spectrometry·2025
Same author

Genetic hypogonadal (Gnrh1<sup>hpg</sup>) mouse model uncovers influence of reproductive axis on maturation of the gut microbiome during puberty.

bioRxiv : the preprint server for biology·2024
Same author

AutoPhy: Automated phylogenetic identification of novel protein subfamilies.

PloS one·2024
Same author

Metabolic dysregulation and gut dysbiosis linked to hyperandrogenism in female mice.

Endocrinology, diabetes & metabolism·2023
Same author

Metagenome-Assembled Genomes from Murine Fecal Microbiomes Dominated by Uncharacterized Bacteria.

Microbiology resource announcements·2023
Same author

Metagenomic Analysis of Microbial Contamination in the U.S. Portion of the Tijuana River Watershed.

International journal of environmental research and public health·2023

Related Experiment Video

Updated: Jul 6, 2025

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.5K

Data-driven mathematical and visualization approaches for removing rare features for Compositional Data Analysis

Adrian Ortiz-Velez1,2, Scott T Kelley1,2

  • 1Biological and Medical Informatics Program, San Diego State University, San Diego, CA 92182, USA.

NAR Genomics and Bioinformatics
|January 8, 2024
PubMed
Summary

CurvCut is a new unsupervised method that removes rare features from sparse biological data, like metagenomics. It uses data-driven breaks to improve statistical power and retain more features without bias.

More Related Videos

Author Spotlight: Emerging Technologies and Advanced Tools for Decoding Metabolomics Data Analysis
07:11

Author Spotlight: Emerging Technologies and Advanced Tools for Decoding Metabolomics Data Analysis

Published on: November 10, 2023

2.4K
Author Spotlight: Integrated Multi-Omics Analysis for Unveiling Multicellular Immune Signatures in Clinical Heart Attack Cohorts
08:51

Author Spotlight: Integrated Multi-Omics Analysis for Unveiling Multicellular Immune Signatures in Clinical Heart Attack Cohorts

Published on: September 20, 2024

1.3K

Related Experiment Videos

Last Updated: Jul 6, 2025

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.5K
Author Spotlight: Emerging Technologies and Advanced Tools for Decoding Metabolomics Data Analysis
07:11

Author Spotlight: Emerging Technologies and Advanced Tools for Decoding Metabolomics Data Analysis

Published on: November 10, 2023

2.4K
Author Spotlight: Integrated Multi-Omics Analysis for Unveiling Multicellular Immune Signatures in Clinical Heart Attack Cohorts
08:51

Author Spotlight: Integrated Multi-Omics Analysis for Unveiling Multicellular Immune Signatures in Clinical Heart Attack Cohorts

Published on: September 20, 2024

1.3K

Area of Science:

  • Bioinformatics
  • Computational Biology
  • Data Science

Background:

  • Sparse feature tables with many rare features are common in large biological datasets, such as metagenomics.
  • Ignoring zero-laden data can bias statistical estimates and reduce the power of downstream analyses.
  • Log-ratio analysis in compositional data is problematic due to undefined log(0).

Purpose of the Study:

  • To introduce CurvCut, an unsupervised, data-driven method for rare-feature removal in sparse biological data.
  • To provide a robust and justifiable approach for handling low-frequency features, addressing limitations of arbitrary removal thresholds.
  • To enable researchers to confidently identify and apply feature removal cutoffs, maximizing feature retention and analytical power.

Main Methods:

  • CurvCut employs two unsupervised methods to identify natural breaks in feature frequency distributions: curvature analysis and the Fisher-Jenks statistical method.
  • The approach incorporates human confirmation for validating identified cutoffs.
  • It is designed to be applicable across diverse biological data types.

Main Results:

  • CurvCut effectively identifies data-specific cutoffs for low-frequency feature removal, maximizing feature retention.
  • The method demonstrates rapid identification of these breaks and generates clear visualizations for user confirmation.
  • Application across different biological data types confirmed its versatility and effectiveness.

Conclusions:

  • CurvCut offers a data-driven, unsupervised solution for rare-feature removal in sparse biological datasets.
  • The method enhances statistical power and reduces bias in downstream analyses by providing justifiable feature removal cutoffs.
  • CurvCut facilitates improved data preprocessing for compositional data analysis and other bioinformatics applications.