Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Quantifying and Rejecting Outliers: The Grubbs Test01:02

Quantifying and Rejecting Outliers: The Grubbs Test

1.7K
Sometimes, a data set can have a recorded numerical observation that greatly  deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier.  To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.7K
Detection of Gross Error: The Q Test01:00

Detection of Gross Error: The Q Test

6.2K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.2K
What Are Outliers?01:12

What Are Outliers?

3.9K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
3.9K
Outliers and Influential Points01:08

Outliers and Influential Points

4.1K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
4.1K
Friedman Two-way Analysis of Variance by Ranks01:21

Friedman Two-way Analysis of Variance by Ranks

240
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
240
Review and Preview01:10

Review and Preview

7.6K
In statistics, several tools are used to interpret the data. Measures of central tendency represent the characteristics of the data, such as mean, median, and mode. Additionally, measures of variance like standard deviation and range are used to find the spread of data from the mean. Relative standing measures the distance between data locations. Commonly used measures of relative standings are percentile, z score, and quartiles.
Percentiles are a type of fractile that partition data into...
7.6K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Effect of icosapent ethyl treatment on colorectal tissue marine omega-3 polyunsaturated fatty acid levels among patients with a history of adenoma: a prospective, single-arm clinical trial.

The American journal of clinical nutrition·2026
Same author

A multi-state survival model to identify risk factors for lethal ovarian cancer.

Cancer epidemiology, biomarkers & prevention : a publication of the American Association for Cancer Research, cosponsored by the American Society of Preventive Oncology·2026
Same author

Associations of alcohol use with expression of stromal markers in benign breast biopsy samples.

British journal of cancer·2026
Same author

Reliability of stromal markers multiplex immunofluorescent staining: pathologist assessment compared to quantitative image analysis.

American journal of cancer research·2026
Same author

Associations of oral contraceptives with expression of CD44, CD24, and ALDH1A1 stem cell markers in women with benign breast biopsies.

Breast cancer research : BCR·2026
Same author

Non-medical drivers of health and imaging patterns in intimate partner violence.

Emergency radiology·2026
Same journal

Methods for incorporating test result information within the high-dimensional propensity score framework: application in UK electronic health record data.

BMC medical research methodology·2026
Same journal

Sparse multi-way DMDC for longitudinal classification in high dimension low sample size data.

BMC medical research methodology·2026
Same journal

Tree-based exploratory identification of predictive biomarkers in non-randomized data.

BMC medical research methodology·2026
Same journal

Comparative evaluation of interrupted time series analytical methods for healthcare quality improvement research: a Monte Carlo simulation study.

BMC medical research methodology·2026
Same journal

Methodological advances in claims-based dementia algorithms: integrating medication and clinical data for medicare populations.

BMC medical research methodology·2026
Same journal

An interpretable XGboost algorithm for predicting 30-day mortality in acute pancreatitis using routine biomarkers.

BMC medical research methodology·2026
See all related articles

Related Experiment Video

Updated: Jul 20, 2025

A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences
00:08

A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences

Published on: September 4, 2019

7.1K

Analytical method for detecting outlier evaluators.

Yujie Wu1, Sharon Curhan2,3, Bernard Rosner1,2

  • 1Department of Biostatistics, Harvard University, Boston, USA.

BMC Medical Research Methodology
|August 1, 2023
PubMed
Summary
This summary is machine-generated.

A new two-stage algorithm effectively identifies outlier evaluators in epidemiologic studies, improving data quality. This method accurately detects inconsistent evaluations, reducing bias and enhancing the reliability of study findings.

Keywords:
EvaluatorFalse discovery rateOutlier detectionQuality controlReviewer

More Related Videos

Denver Papillae Protocol for Objective Analysis of Fungiform Papillae
10:50

Denver Papillae Protocol for Objective Analysis of Fungiform Papillae

Published on: June 8, 2015

14.4K
Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios
06:02

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Published on: October 6, 2020

2.3K

Related Experiment Videos

Last Updated: Jul 20, 2025

A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences
00:08

A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences

Published on: September 4, 2019

7.1K
Denver Papillae Protocol for Objective Analysis of Fungiform Papillae
10:50

Denver Papillae Protocol for Objective Analysis of Fungiform Papillae

Published on: June 8, 2015

14.4K
Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios
06:02

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Published on: October 6, 2020

2.3K

Area of Science:

  • Epidemiology
  • Biostatistics
  • Data Quality Assurance

Background:

  • Epidemiologic studies rely on evaluator measurements for data quality, which can be compromised by outliers.
  • Existing statistical methods for measurement error adjustment often depend on unverifiable assumptions, potentially leading to biased results.
  • There is a need for methods to detect outlier evaluators during data collection to improve overall data integrity.

Purpose of the Study:

  • To propose and evaluate a novel two-stage algorithm for detecting outlier evaluators in observational studies.
  • To enhance the reliability of data collected by evaluators in large-scale health studies.
  • To provide a flexible method for improving data quality at the source.

Main Methods:

  • A two-stage algorithm was developed: the first stage fits a regression model to estimate evaluator effects, and the second stage uses hypothesis testing to identify outliers.
  • The method considers both the statistical power of individual tests and the false discovery rate (FDR) across all tests.
  • The algorithm was evaluated through an extensive simulation study and demonstrated using data from the Conservation of Hearing Study.

Main Results:

  • The simulation study confirmed the algorithm's ability to accurately detect true outlier evaluators.
  • The proposed method showed a lower tendency to incorrectly flag valid 'normal' evaluators compared to other approaches.
  • The application to the Conservation of Hearing Study successfully identified potential outlier audiologists.

Conclusions:

  • The developed two-stage outlier detection algorithm offers a flexible and effective approach to identify evaluators with consistently high or low measurements.
  • Implementing this algorithm can significantly improve data quality during the data collection phase of epidemiologic studies.
  • This method contributes to more reliable estimation of associations between exposures and outcomes in health research.