Related Experiment Video
Updated: Jul 9, 2026
07:05
Applying Hyperspectral Reflectance Imaging to Investigate the Palettes and the Techniques of Painters
Published on: June 18, 2021
Individual and joint expert judgments as reference standards in artifact detection.
Marion Verduijn1, Niels Peek, Nicolette F de Keizer
1Department of Medical Informatics, Academic Medical Center, Amsterdam, the Netherlands. m.verduijn@amc.uva.nl
Journal of the American Medical Informatics Association : JAMIA
|December 22, 2007
Summary
Clinical experts show varied agreement on identifying monitoring data artifacts. Using joint expert judgments to develop automatic artifact filters improves their performance, making them more reliable for clinical data analysis.
Area of Science:
- Biomedical Engineering
- Clinical Data Analysis
- Signal Processing
Background:
- Accurate interpretation of physiological monitoring data is crucial for patient care.
- Automated artifact detection and filtering are essential for reliable data analysis.
- Variability in expert judgment can impact the development of robust algorithms.
Purpose of the Study:
- To assess inter-expert agreement on identifying artifacts in physiological monitoring data.
- To evaluate the impact of different expert judgment standards (individual vs. joint) on artifact filter performance.
- To compare the generalizability of artifact filters developed using individual versus consensus expert input.
Main Methods:
- Collected individual judgments from four physicians on 30 time series of mean arterial blood pressure (ABPm), central venous pressure (CVP), and heart rate (HR).
- Derived majority vote and consensus judgments from individual expert inputs.
- Tuned and evaluated three existing artifact filtering methods using individual and joint expert judgments.
- Quantified interrater agreement using positive specific agreement (PSA) and filter performance using sensitivity and positive predictive value (PPV).
Main Results:
- Inter-expert agreement (PSA) varied (0.33-0.85), with higher agreement for CVP data.
- Filters developed using individual expert judgments showed moderate generalization (Sensitivity: 0.40-0.80, PPV: 0.57-0.86).
- Artifact filters demonstrated significantly higher performance when tuned with joint expert judgments across all monitored variables.
Conclusions:
- Significant disagreement exists among clinical experts in identifying data artifacts.
- Joint reference standards derived from multiple experts are recommended for developing more effective automatic artifact filters.
- Utilizing consensus-based tuning enhances the reliability and generalizability of physiological signal processing algorithms.
Related Concept Videos
01:18Effects of EDTA on End-Point Detection Methods
Different methods, such as visual observance of metal-ion indicators, spectroscopic techniques, and potentiometric methods, can determine the endpoint of an EDTA titration.
In the visual method, metal-ion indicators (metallochromic dyes), which have distinct colors in their free and complex forms, are added to the mixture to signal the titration's end point. They form stable complexes with metal ions, but these complexes are weaker than the corresponding metal–EDTA complexes. As a result, EDTA...
In the visual method, metal-ion indicators (metallochromic dyes), which have distinct colors in their free and complex forms, are added to the mixture to signal the titration's end point. They form stable complexes with metal ions, but these complexes are weaker than the corresponding metal–EDTA complexes. As a result, EDTA...
01:20Structural Classification of Joints
Joints, also known as articulations, are classified based on their structural characteristics, i.e., based on whether the articulating surfaces of the adjacent bones are directly connected by fibrous connective tissue or cartilage, or whether the articulating surfaces contact each other within a fluid-filled joint cavity. These differences serve to divide the joints of the body into three structural classifications.
A fibrous joint is where the adjacent bones are united by fibrous connective...
A fibrous joint is where the adjacent bones are united by fibrous connective...
02:46Uncertainty in Measurement: Reading Instruments
Counting is the type of measurement that is free from uncertainty, provided the number of objects being counted does not change during the process. Such measurements result in exact numbers. By counting the eggs in a carton, for instance, one can determine exactly how many eggs are there in the carton. Similarly, the numbers of defined quantities are also exact. For example, 1 foot is exactly 12 inches, 1 inch is exactly 2.54 centimeters, and 1 gram is exactly 0.001 kilograms. Quantities...
01:05Difference from Background: Limit of Detection
The limit of detection (LOD) is the smallest amount of analyte that can be distinguished from the background noise. The LOD value corresponds to the concentration at which the analyte signal is three times larger than the standard deviation of the blank signal. Below this value, the analyte signal cannot be differentiated from the background noise. It is calculated by dividing the calibration slope by 3 times the standard deviation of the blank signals.
The LOD indicates the presence or absence...
The LOD indicates the presence or absence...
01:00Detection of Gross Error: The Q Test
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...