Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Detection of Gross Error: The Q Test01:00

Detection of Gross Error: The Q Test

When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
Group Design02:01

Group Design

The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between the two are due to...
Bonferroni Test01:10

Bonferroni Test

The Bonferroni test is a statistical test named after Carlo Emilio Bonferroni, an Italian mathematician best known for Bonferroni inequalities. This statistical test is a type of multiple comparison test to determine which means are different than the rest. Bonferroni test can minimize the Type 1 error by reducing the significance level alpha, which otherwise increases with sample pairs.
The means of different samples are first paired in all possible combinations.
The null hypothesis of the...
Quantifying and Rejecting Outliers: The Grubbs Test01:02

Quantifying and Rejecting Outliers: The Grubbs Test

Sometimes, a data set can have a recorded numerical observation that greatly  deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier.  To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This number is...
Surveys02:16

Surveys

Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
Strategies for Assessing and Addressing Confounding01:25

Strategies for Assessing and Addressing Confounding

Confounding is a critical issue in epidemiological studies, often leading to misleading conclusions about associations between exposures and outcomes. It occurs when the relationship between the exposure and the outcome is mixed with the effects of other factors that influence the outcome. Given that, addressing confounding is of high importance for drawing accurate inferences in research.
Confounding can be addressed at both the design phase of a study and through analytical methods after data...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Metformin modifies hormone changes associated with androgen deprivation therapy for prostate cancer.

Endocrine oncology (Bristol, England)·2026
Same author

Teaching Clinical Reasoning in Health Care Professions Learners Using AI-Generated Script Concordance Tests: Mixed Methods Formative Evaluation.

JMIR formative research·2025
Same author

Evaluation of the "Resident as teacher" curriculum: a needs assessment in medical education at a large academic institution.

MedEdPublish (2016)·2025
Same author

Prostate-Specific Membrane Antigen PET-Guided Intensification of Salvage Radiotherapy After Radical Prostatectomy: A Phase 2 Randomized Clinical Trial.

JAMA oncology·2025
Same author

A Retrospective Study on Prognostic Factors and Systemic Treatments of Refractory Meningiomas.

Current oncology (Toronto, Ont.)·2025
Same author

Reply by Authors.

The Journal of urology·2025

Related Experiment Video

Updated: Jun 4, 2026

Irrelevant Stimuli and Action Control: Analyzing the Influence of Ignored Stimuli via the Distractor-Response Binding Paradigm
12:12

Irrelevant Stimuli and Action Control: Analyzing the Influence of Ignored Stimuli via the Distractor-Response Binding Paradigm

Published on: May 14, 2014

Optimization of answer keys for script concordance testing: should we exclude deviant panelists, deviant responses,

Robert Gagnon1, Stuart Lubarsky, Carole Lambert

  • 1CPASS, Faculty of Medicine, University of Montreal, Centre-Ville Station, QC, Canada.

Advances in Health Sciences Education : Theory and Practice
|February 3, 2011
PubMed
Summary
This summary is machine-generated.

Removing outlier panelists or answers from Script Concordance Tests (SCT) minimally impacts score reliability, especially with larger panels. The distance-from-mode method is recommended for excluding deviant answers if necessary.

More Related Videos

A Naturalistic Setup for Presenting Real People and Live Actions in Experimental Psychology and Cognitive Neuroscience Studies
07:43

A Naturalistic Setup for Presenting Real People and Live Actions in Experimental Psychology and Cognitive Neuroscience Studies

Published on: August 4, 2023

Problem-Solving Before Instruction (PS-I): A Protocol for Assessment and Intervention in Students with Different Abilities
10:26

Problem-Solving Before Instruction (PS-I): A Protocol for Assessment and Intervention in Students with Different Abilities

Published on: September 11, 2021

Related Experiment Videos

Last Updated: Jun 4, 2026

Irrelevant Stimuli and Action Control: Analyzing the Influence of Ignored Stimuli via the Distractor-Response Binding Paradigm
12:12

Irrelevant Stimuli and Action Control: Analyzing the Influence of Ignored Stimuli via the Distractor-Response Binding Paradigm

Published on: May 14, 2014

A Naturalistic Setup for Presenting Real People and Live Actions in Experimental Psychology and Cognitive Neuroscience Studies
07:43

A Naturalistic Setup for Presenting Real People and Live Actions in Experimental Psychology and Cognitive Neuroscience Studies

Published on: August 4, 2023

Problem-Solving Before Instruction (PS-I): A Protocol for Assessment and Intervention in Students with Different Abilities
10:26

Problem-Solving Before Instruction (PS-I): A Protocol for Assessment and Intervention in Students with Different Abilities

Published on: September 11, 2021

Area of Science:

  • Medical Education
  • Psychometrics
  • Radiation Oncology

Background:

  • The Script Concordance Test (SCT) assesses clinical reasoning variability among practitioners.
  • Deviant responses in SCT scoring can introduce measurement error, potentially affecting score reliability.
  • Optimizing SCT scoring methods is crucial for accurate assessment of clinical judgment.

Purpose of the Study:

  • To investigate the psychometric impact of excluding deviant panelists or answers from SCT scoring.
  • To propose methods for excluding deviant panelists or answers to enhance SCT reliability.
  • To evaluate the effectiveness of different outlier exclusion strategies on SCT psychometrics.

Main Methods:

  • Examined three methods for reducing panel response variability in an SCT: outlier panelists, distance-from-mode, and judgment-by-experts.
  • Assessed the effects of these methods on score reliability, score correlations, and between-group effect sizes.
  • Compared results using large (n=45) and small (n=15) panel sizes.

Main Results:

  • With large panels (n=45), outlier exclusion methods showed no significant effect on SCT reliability, correlation, or effect size.
  • With smaller panels (n=15), methods had no significant effect on reliability or correlation, but varied effect sizes.
  • Measurement error from deviant responses is negligible with large panels (>15).

Conclusions:

  • Panel size is a key factor in the impact of outlier exclusion on SCT psychometrics.
  • Excluding deviant panelists has a negligible effect on SCT reliability for sufficiently large panels.
  • The distance-from-mode strategy is recommended for excluding deviant answers when necessary.