Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Quantifying and Rejecting Outliers: The Grubbs Test01:02

Quantifying and Rejecting Outliers: The Grubbs Test

1.4K
Sometimes, a data set can have a recorded numerical observation that greatly  deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier.  To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.4K
Cluster Sampling Method01:20

Cluster Sampling Method

11.6K
Appropriate sampling methods ensure that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
11.6K
Sampling Plans01:23

Sampling Plans

162
Sampling is a crucial step in analytical chemistry, allowing researchers to collect representative data from a large population. Common sampling methods include random, judgmental, systematic, stratified, and cluster sampling.
Random sampling is a method where each member of the population has an equal chance of being selected for the sample. It involves selecting individuals randomly, often using random number generators or lottery-type methods. For example, when analyzing the properties of a...
162
Detection of Gross Error: The Q Test01:00

Detection of Gross Error: The Q Test

5.0K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
5.0K
Expected Frequencies in Goodness-of-Fit Tests01:19

Expected Frequencies in Goodness-of-Fit Tests

2.5K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n)  to the number of categories (k).
2.5K
Comparing the Survival Analysis of Two or More Groups01:20

Comparing the Survival Analysis of Two or More Groups

110
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
110

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

60 years of OR in Slovenia: development from a first conference to a vibrant community.

Central European journal of operations research·2023
Same author

Collective dynamics of phase-repulsive oscillators solves graph coloring problem.

Chaos (Woodbury, N.Y.)·2020
Same author

Author Correction: Towards a data-integrated cell.

Nature communications·2019
Same author

Towards a data-integrated cell.

Nature communications·2019
Same author

Community structure and the evolution of interdisciplinarity in Slovenia's scientific collaboration network.

PloS one·2014

Related Experiment Video

Updated: May 21, 2025

Large-scale Reconstructions and Independent, Unbiased Clustering Based on Morphological Metrics to Classify Neurons in Selective Populations
12:27

Large-scale Reconstructions and Independent, Unbiased Clustering Based on Morphological Metrics to Classify Neurons in Selective Populations

Published on: February 15, 2017

6.9K

Ground truth clustering is not the optimum clustering.

Lucia Absalom Bautista1, Timotej Hrga2, Janez Povh3,4

  • 1University of Sevilla, C. San Fernando 4, Seville, 41004, Spain.

Scientific Reports
|March 18, 2025
PubMed
Summary

Optimal data clustering solutions often differ from ground truth, yet can yield superior intrinsic quality. Alignment improves when ground truth clusters are well-separated convex shapes.

Keywords:
Extrinsic measuresGround truth clusteringIntrinsic measuresMinimum sum-of-squares clustering

More Related Videos

Computation of Atmospheric Concentrations of Molecular Clusters from ab initio Thermochemistry
12:11

Computation of Atmospheric Concentrations of Molecular Clusters from ab initio Thermochemistry

Published on: April 8, 2020

8.1K
Sampling Soils in a Heterogeneous Research Plot
07:11

Sampling Soils in a Heterogeneous Research Plot

Published on: January 7, 2019

34.4K

Related Experiment Videos

Last Updated: May 21, 2025

Large-scale Reconstructions and Independent, Unbiased Clustering Based on Morphological Metrics to Classify Neurons in Selective Populations
12:27

Large-scale Reconstructions and Independent, Unbiased Clustering Based on Morphological Metrics to Classify Neurons in Selective Populations

Published on: February 15, 2017

6.9K
Computation of Atmospheric Concentrations of Molecular Clusters from ab initio Thermochemistry
12:11

Computation of Atmospheric Concentrations of Molecular Clusters from ab initio Thermochemistry

Published on: April 8, 2020

8.1K
Sampling Soils in a Heterogeneous Research Plot
07:11

Sampling Soils in a Heterogeneous Research Plot

Published on: January 7, 2019

34.4K

Area of Science:

  • Data Science
  • Computational Statistics
  • Machine Learning

Background:

  • Data clustering is crucial but challenging.
  • Minimum Sum-of-Squares Clustering (MSSC) aims to minimize point-to-centroid distances.
  • MSSC is NP-hard, but solvers exist for optimal solutions.

Purpose of the Study:

  • To obtain and evaluate optimal MSSC solutions using the SOS-SDP solver.
  • To compare optimal clusterings against ground truth clusterings on various datasets.
  • To assess the quality of optimal clusterings using extrinsic and intrinsic measures.

Main Methods:

  • Utilized the SOS-SDP solver, a branch-and-bound algorithm based on semidefinite programming.
  • Obtained optimal MSSC solutions for diverse datasets with known ground truth.
  • Evaluated clustering alignment with six extrinsic and three intrinsic measures.

Main Results:

  • Optimal clusterings frequently diverge from ground truth clusterings.
  • Optimal clusterings often demonstrate superior intrinsic quality compared to ground truth.
  • High alignment observed when ground truth clusters are well-separated convex shapes like ellipsoids.

Conclusions:

  • Optimal MSSC solutions may not always match human-defined ground truth.
  • Intrinsic cluster quality can be higher in mathematically optimal solutions.
  • The geometric properties of data clusters influence the agreement between optimal and ground truth solutions.