Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

What Are Outliers?

What Are Outliers?

Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...

Outliers and Influential Points

Outliers and Influential Points

An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...

Naturalistic Observations

Naturalistic Observations

If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances...

Self-Evaluation: Self-Enhancement and Self-Verification

Self-Evaluation: Self-Enhancement and Self-Verification

Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...

Review and Preview

Review and Preview

In statistics, several tools are used to interpret the data. Measures of central tendency represent the characteristics of the data, such as mean, median, and mode. Additionally, measures of variance like standard deviation and range are used to find the spread of data from the mean. Relative standing measures the distance between data locations. Commonly used measures of relative standings are percentile, z score, and quartiles.
Percentiles are a type of fractile that partition data into...

Quantifying and Rejecting Outliers: The Grubbs Test

Quantifying and Rejecting Outliers: The Grubbs Test

Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Identifying barriers and facilitators to use of the Canadian emergency department best practices checklist for skin and soft tissue infections.

CJEM·2026

Same author

Beyond Individualism and Independence: Exploring Collectivism and Interdependence as Paradigms of Healthcare and Health Professions Education.

Perspectives on medical education·2025

Same author

Barriers and facilitators to use of the HEARTRISK6 Scale for acute heart failure presentations in the emergency department: a qualitative study.

CJEM·2025

Same author

Home Field Advantage? Comparing the Quality of EPA Observations Completed On- vs Off-Service.

Journal of graduate medical education·2025

Same author

Hidden beneath the surface: what are the knowledge and skills for successful transition to practice.

CJEM·2025

Same author

Unveiling the shadows: A qualitative inquiry into burnout perceptions among pre-clerkship students in Canadian medical education.

Medical teacher·2025

Same journal

When what we can count is all that counts.

Medical teacher·2026

Same journal

Reframing interprofessional competencies through Ubuntu: A decolonial proposal.

Medical teacher·2026

Same journal

Uncovering the economic costs of rotation in postgraduate clinical training: A UK case study using a novel methodology.

Medical teacher·2026

Same journal

The professional identity formation of undergraduate and postgraduate medical students: An umbrella review.

Medical teacher·2026

Same journal

'Place' and clinical reasoning development in context: An ethnographic case study.

Medical teacher·2026

Same journal

Fit for context: Faculty perspectives on mentorship and culture in Sub-saharan Africa.

Medical teacher·2026

See all related articles

Search research articles

Related Experiment Video

Updated: Jan 9, 2026

Assessing the Coherence of Parents' Short Narratives Regarding their Child Using the Five-Minute Speech Sample Procedure

Assessing the Coherence of Parents' Short Narratives Regarding their Child Using the Five-Minute Speech Sample Procedure

Published on: September 19, 2019

Do outlier assessors provide useful narrative comments?

Sebastian Dewhirst¹, Nora D Szabo¹, Andrew K Hall¹

¹Department of Emergency Medicine, University of Ottawa, Ottawa, Canada.

Medical Teacher

|December 10, 2025

Summary

This summary is machine-generated.

Outlier assessors, whether lenient or range-restricted, provide low-utility assessments. Targeted interventions are needed to improve the quality of both numeric and narrative feedback from these educators.

Keywords:

Narrative assessment assessor stringency range restriction

More Related Videos

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Published on: September 27, 2020

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Published on: October 6, 2020

Related Experiment Videos

Last Updated: Jan 9, 2026

Assessing the Coherence of Parents' Short Narratives Regarding their Child Using the Five-Minute Speech Sample Procedure

Assessing the Coherence of Parents' Short Narratives Regarding their Child Using the Five-Minute Speech Sample Procedure

Published on: September 19, 2019

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Published on: September 27, 2020

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Evaluating Usability Aspects of a Mixed Reality Solution for Immersive Analytics in Industry 4.0 Scenarios

Published on: October 6, 2020

Area of Science:

Medical Education
Assessment Quality
Emergency Medicine

Background:

Assessor variability, including stringency/leniency and range restriction, can impact the utility of educational assessments.
The quality of narrative comments from outlier assessors is not well understood.

Purpose of the Study:

To evaluate the utility of assessment scores provided by outlier assessors (stringent/lenient and range-restricted).
To determine if outlier status and range restriction correlate with lower assessment quality.

Main Methods:

Utilized end-of-shift assessments from an academic emergency department, scored using the Quality of Assessment of Learning (QuAL) score.
Quantified assessor stringency/leniency and range restriction using mean-delta and standard deviation.
Compared mean QuAL scores of outlier vs. non-outlier assessors using T-tests and linear regression.

Main Results:

Identified 20 outlier assessors (10 stringent, 10 lenient) among 81 total assessors.
Mean QuAL scores for non-outliers (4.0) were significantly higher than for lenient outliers (3.1).
Range restriction was negatively correlated with mean QuAL scores (R²=0.11, p=0.003).

Conclusions:

Outlier lenient and range-restricted assessors provide lower-quality narrative assessments.
Assessors with both numeric and narrative assessments of limited utility require targeted interventions.
Improving assessor quality is crucial for effective medical education and learner development.