Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Data Collection by Observations01:08

Data Collection by Observations

15.0K
Data collection refers to a systematic way of obtaining, observing, measuring, and analyzing accurate information. Observational studies are one of the most widely used methods of data collection. It involves collecting data by observing the behavior and physical characteristics of a sample without making any modifications to the sample.
An astronomer viewing the motion and brightness of stars in the sky and recording the data is an example of observational data collection. A botanist recording...
15.0K
What Are Outliers?01:12

What Are Outliers?

5.1K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
5.1K
Outliers and Influential Points01:08

Outliers and Influential Points

6.3K
An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...
6.3K
Statistical Software for Data Analysis and Clinical Trials01:12

Statistical Software for Data Analysis and Clinical Trials

1.5K
Statistical software is pivotal in data analysis and clinical trials by providing tools to analyze data, draw conclusions, and make predictions. These software packages range from simple data management applications to complex analytical platforms, supporting various statistical tests, models, and simulation techniques. Their significance lies in their ability to handle vast amounts of data with precision and efficiency, enabling researchers to validate hypotheses, identify trends, and make...
1.5K
Encoding01:19

Encoding

845
Information enters the brain through encoding, which is the input of information into the memory system. Once sensory information is received from the environment, the brain labels or codes it. The information is then organized with similar information and connected to existing concepts. Encoding occurs through automatic processing and effortful processing.
Automatic processing involves the encoding of details like time, space, frequency, and the meaning of words, usually done without conscious...
845
Quantifying and Rejecting Outliers: The Grubbs Test01:02

Quantifying and Rejecting Outliers: The Grubbs Test

4.1K
Sometimes, a data set can have a recorded numerical observation that greatly  deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier.  To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
4.1K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

i2b2-ML: module to facilitate machine learning in the informatics for integrating biology and the bedside platform.

JAMIA open·2026
Same author

Long COVID Persistence and Surveillance Gaps Across 58 US Hospitals.

JAMA network open·2026
Same author

Beyond human-initiated screening: the imperative for fully autonomous brain health surveillance.

Brain : a journal of neurology·2026
Same author

Developing a community-prioritized research agenda on primary care strategies for minimizing the health impacts of heat and poor air quality events: protocol.

Frontiers in climate·2026
Same author

Understanding data differences across the ENACT federated research network.

JAMIA open·2026
Same author

Paxlovid shows organ-specific and age-specific impacts on risk of developing post-acute sequelae of COVID-19.

Communications medicine·2026

Related Experiment Video

Updated: Jan 30, 2026

Supervised Machine Learning for Semi-Quantification of Extracellular DNA in Glomerulonephritis
09:16

Supervised Machine Learning for Semi-Quantification of Extracellular DNA in Glomerulonephritis

Published on: June 18, 2020

7.3K

Semi-supervised encoding for outlier detection in clinical observation data.

Hossein Estiri1, Shawn N Murphy1

  • 1Harvard Medical School, Boston, MA, USA; Massachusetts General Hospital, Boston, MA, USA; Partners Healthcare, Somerville, MA, USA.

Computer Methods and Programs in Biomedicine
|January 20, 2019
PubMed
Summary

A novel semi-supervised encoding method effectively detects implausible outliers in Electronic Health Record (EHR) data. This approach enhances data quality by identifying unreliable laboratory and vital sign observations.

Keywords:
Data qualityElectronic Health RecordsEncodingNeural NetworksOutlier detectionSemi-supervised encoding

More Related Videos

Analyzing Mitochondrial Morphology Through Simulation Supervised Learning
12:06

Analyzing Mitochondrial Morphology Through Simulation Supervised Learning

Published on: March 3, 2023

4.7K
Author Spotlight: Insights into Remotely Supervised Neuromodulation Procedure for Phantom Limb Pain
06:13

Author Spotlight: Insights into Remotely Supervised Neuromodulation Procedure for Phantom Limb Pain

Published on: March 1, 2024

1.8K

Related Experiment Videos

Last Updated: Jan 30, 2026

Supervised Machine Learning for Semi-Quantification of Extracellular DNA in Glomerulonephritis
09:16

Supervised Machine Learning for Semi-Quantification of Extracellular DNA in Glomerulonephritis

Published on: June 18, 2020

7.3K
Analyzing Mitochondrial Morphology Through Simulation Supervised Learning
12:06

Analyzing Mitochondrial Morphology Through Simulation Supervised Learning

Published on: March 3, 2023

4.7K
Author Spotlight: Insights into Remotely Supervised Neuromodulation Procedure for Phantom Limb Pain
06:13

Author Spotlight: Insights into Remotely Supervised Neuromodulation Procedure for Phantom Limb Pain

Published on: March 1, 2024

1.8K

Area of Science:

  • Biomedical Informatics
  • Data Science
  • Machine Learning

Background:

  • Electronic Health Record (EHR) data frequently contain implausible observations, particularly in laboratory tests and vital signs.
  • These unreliable data points can compromise clinical decision-making and data analysis.
  • Outlier detection methods offer a cost-effective strategy for identifying such data anomalies.

Purpose of the Study:

  • To evaluate the effectiveness of a semi-supervised encoding approach, termed super-encoding, for detecting outliers in EHR observation data.
  • To construct non-linear data distributions from EHR observations to identify non-conforming data points.
  • To assess the impact of demographic features and data sampling on outlier detection precision.

Main Methods:

  • Two hypotheses were tested: (1) demographic features improve outlier detection precision, and (2) sampling small data subsets enhances detection by reducing noise.
  • 492 encoder configurations were applied to 30 EHR observation datasets (laboratory tests, vital signs).
  • Experiments utilized non-parametric hypothesis testing and experimental design on data from the Research Patient Data Registry (RPDR).

Main Results:

  • The semi-supervised encoding approach (super-encoding) demonstrated superior performance compared to conventional autoencoders in outlier detection.
  • Incorporating patient age as a feature slightly improved outlier detection accuracy.
  • The top-performing configuration used a semi-supervised encoder with observation value as the sole feature, a single hidden layer, and was trained on 1% of the data with 1% reconstruction error, achieving a Youden's J index > 0.9999 for all observation types.

Conclusions:

  • Encoding methods show significant promise for outlier detection in large-scale EHR data repositories, addressing the complexity of human observations and data variability.
  • The super-encoding approach offers a robust solution for improving the quality and reliability of EHR data.
  • The study highlights the potential of machine learning for enhancing the utility of routinely collected health data.