Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Modern Molecular Taxonomy01:29

Modern Molecular Taxonomy

74
Advancements in molecular biology have revolutionized the identification and characterization of bacteria, with multiple methods leveraging DNA sequencing for enhanced precision. As sequencing technologies improve and costs decline, these approaches are increasingly used in clinical, environmental, and evolutionary studies.Multilocus Sequence Typing (MLST) examines several housekeeping genes, essential chromosomal genes encoding cellular functions, to distinguish strains. Approximately...
74
Applications of Molecular Taxonomy01:20

Applications of Molecular Taxonomy

56
Molecular taxonomy has revolutionized the understanding and classification of bacteria, providing precise insights into their diversity, evolutionary relationships, and ecological roles. By utilizing molecular techniques such as DNA sequencing and fingerprinting, researchers have made significant strides in various fields related to bacterial studies.Resolving Taxonomic AmbiguitiesMolecular taxonomy has been instrumental in distinguishing closely related bacterial species initially thought to...
56
MALDI-TOF Mass Spectrometry01:19

MALDI-TOF Mass Spectrometry

5.0K
Mass spectrometry is a powerful characterization technique that can identify and separate a wide variety of compounds ranging from chemical to biological entities, based on their mass-to-charge ratio (m/z). The instruments that allow this detection, known as mass spectrometers, have three components: an ion source, a mass analyzer, and a detector. These spectrometers differ based on the nature of their ion source and analyzers.
Matrix-assisted laser desorption ionization (MALDI) is a commonly...
5.0K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

When to Adjust for Multiple Testing: A Unifying Guiding Principle.

Biometrical journal. Biometrische Zeitschrift·2026
Same author

Score Matching for Differential Abundance Testing of Compositional High-Throughput Sequencing Data.

Statistics in medicine·2026
Same author

STrategies for developing REseArch Methods guidance (STREAM): Protocol.

Journal of clinical epidemiology·2026
Same author

The statistical software revolution in pharmaceutical development: challenges and opportunities in open source.

Drug discovery today·2026
Same author

On "Confirmatory" Methodological Research in Statistics and Related Fields.

Statistics in medicine·2025
Same author

ChatGPT as a Tool for Biostatisticians: A Tutorial on Applications, Opportunities, and Limitations.

Statistics in medicine·2025

Related Experiment Video

Updated: Aug 15, 2025

Divergence of Root Microbiota in Different Habitats based on Weighted Correlation Networks
09:49

Divergence of Root Microbiota in Different Habitats based on Weighted Correlation Networks

Published on: September 25, 2021

4.4K

Over-optimism in unsupervised microbiome analysis: Insights from network learning and clustering.

Theresa Ullmann1,2, Stefanie Peschel3,4, Philipp Finger1

  • 1Institute for Medical Information Processing, Biometry, and Epidemiology, Ludwig-Maximilians-Universität München, München, Germany.

Plos Computational Biology
|January 6, 2023
PubMed
Summary

Researchers often face over-optimism when analyzing microbiome data. Trying multiple methods and reporting only the best results leads to inflated discovery performance, hindering reliable microbiome research.

More Related Videos

Microbiota Analysis Using Two-step PCR and Next-generation 16S rRNA Gene Sequencing
11:22

Microbiota Analysis Using Two-step PCR and Next-generation 16S rRNA Gene Sequencing

Published on: October 15, 2019

28.8K
Assisted Selection of Biomarkers by Linear Discriminant Analysis Effect Size LEfSe in Microbiome Data
04:57

Assisted Selection of Biomarkers by Linear Discriminant Analysis Effect Size LEfSe in Microbiome Data

Published on: May 16, 2022

16.1K

Related Experiment Videos

Last Updated: Aug 15, 2025

Divergence of Root Microbiota in Different Habitats based on Weighted Correlation Networks
09:49

Divergence of Root Microbiota in Different Habitats based on Weighted Correlation Networks

Published on: September 25, 2021

4.4K
Microbiota Analysis Using Two-step PCR and Next-generation 16S rRNA Gene Sequencing
11:22

Microbiota Analysis Using Two-step PCR and Next-generation 16S rRNA Gene Sequencing

Published on: October 15, 2019

28.8K
Assisted Selection of Biomarkers by Linear Discriminant Analysis Effect Size LEfSe in Microbiome Data
04:57

Assisted Selection of Biomarkers by Linear Discriminant Analysis Effect Size LEfSe in Microbiome Data

Published on: May 16, 2022

16.1K

Area of Science:

  • Microbiome research
  • Computational biology
  • Statistical analysis

Background:

  • Unsupervised analysis of microbiome data, including network analysis and clustering, is increasingly popular.
  • A wide array of statistical and computational methods exist, posing a challenge for researchers in selecting appropriate methods.
  • Selective reporting of optimal results can lead to over-optimism and non-replicable findings, hindering scientific progress.

Purpose of the Study:

  • To quantify over-optimism effects in unsupervised microbiome analysis.
  • To model a hypothetical researcher's approach to unsupervised tasks and evaluate method selection biases.
  • To highlight the importance of validation and replication in microbiome data analysis.

Main Methods:

  • Simulated a hypothetical researcher performing four unsupervised tasks: clustering bacterial genera, hub detection, differential network analysis, and sample clustering.
  • Used the American Gut Project dataset, randomly splitting it into discovery and validation sets multiple times.
  • Tested multiple method combinations for each task on discovery data, selecting the best-performing combination based on predefined criteria, and then applied it to validation data.

Main Results:

  • Over-optimism effects were observed across all four unsupervised research tasks.
  • Results on validation datasets were consistently worse than those on discovery datasets, averaged over multiple random splits.
  • The chosen 'best' method combinations showed a performance decrease when applied to unseen validation data.

Conclusions:

  • Selective reporting and method over-fitting in unsupervised microbiome analysis lead to over-optimism.
  • Validation and replication are crucial for obtaining reliable and generalizable results in microbiome research.
  • The issue of over-optimism extends beyond statistical testing and significance fishing, impacting unsupervised analyses.