Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Self-Report Tests of Personality01:22

Self-Report Tests of Personality

262
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
262

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

A Conceptual Model for Embedding Automated Assessments in Graduate Medical Education.

Journal of graduate medical education·2026
Same author

Into the light: the rise of shadowbox simulation to illuminate decision-making.

Advances in simulation (London, England)·2026
Same author

A systematic literature search and narrative synthesis of economic drivers in hospitalizations for heart failure with preserved or mildly reduced ejection fraction in the United States.

The American journal of managed care·2026
Same author

Cerebrospinal fluid heparan sulfate as a biomarker for neuronopathic mucopolysaccharidoses: Rationale and regulatory challenges.

Molecular genetics and metabolism·2026
Same author

The delicate dance of debriefing: exploring how behavioural marker systems influence the socio-emotional dynamics of simulation practice.

Advances in simulation (London, England)·2026
Same author

Emotions on Call: Making a Hidden Curriculum Visible.

The clinical teacher·2026
Same journal

Self-efficacy in mass casualty incident training using mixed reality.

Advances in simulation (London, England)·2026
Same journal

Artificial intelligence to support debriefing in simulation-based healthcare education: a scoping review.

Advances in simulation (London, England)·2026
Same journal

Impact of interprofessional blended learning with low-fidelity simulation on neonatal resuscitation skills among interns: a quasi-experimental study in Tanzania.

Advances in simulation (London, England)·2026
Same journal

Performance using a visuo-haptic surgical simulator is affected by age.

Advances in simulation (London, England)·2026
Same journal

Strengthening simulation design and cultural safe practice: end-user evaluation of the quality simulation assurance framework.

Advances in simulation (London, England)·2026
Same journal

AI integration in healthcare simulation debriefing: support or substitute?

Advances in simulation (London, England)·2026
See all related articles

Related Experiment Video

Updated: May 7, 2025

Use of a Video Scoring Anchor for Rapid Serial Assessment of Social Communication in Toddlers
09:16

Use of a Video Scoring Anchor for Rapid Serial Assessment of Social Communication in Toddlers

Published on: March 14, 2018

10.2K

Beyond reliability: assessing rater competence when using a behavioural marker system.

Samantha Eve Smith1, Scott McColgan-Smith2, Fiona Stewart2

  • 1Centre for Medical Education, University of Dundee, Dundee, UK. SSmith004@dundee.ac.uk.

Advances in Simulation (London, England)
|December 31, 2024
PubMed
Summary
This summary is machine-generated.

Experienced faculty demonstrated acceptable inter-rater reliability with the pharmacists' behavioural skills (PhaBS) system, but not all raters were competent. Applying rater competence frameworks is crucial for fair and accurate behavioural skills assessments in healthcare education.

More Related Videos

Detecting Behavioral Deficits in Rats After Traumatic Brain Injury
07:54

Detecting Behavioral Deficits in Rats After Traumatic Brain Injury

Published on: January 30, 2018

17.8K
Operant Procedures for Assessing Behavioral Flexibility in Rats
08:30

Operant Procedures for Assessing Behavioral Flexibility in Rats

Published on: February 15, 2015

20.7K

Related Experiment Videos

Last Updated: May 7, 2025

Use of a Video Scoring Anchor for Rapid Serial Assessment of Social Communication in Toddlers
09:16

Use of a Video Scoring Anchor for Rapid Serial Assessment of Social Communication in Toddlers

Published on: March 14, 2018

10.2K
Detecting Behavioral Deficits in Rats After Traumatic Brain Injury
07:54

Detecting Behavioral Deficits in Rats After Traumatic Brain Injury

Published on: January 30, 2018

17.8K
Operant Procedures for Assessing Behavioral Flexibility in Rats
08:30

Operant Procedures for Assessing Behavioral Flexibility in Rats

Published on: February 15, 2015

20.7K

Area of Science:

  • Healthcare education
  • Pharmacy practice
  • Behavioural sciences

Background:

  • Behavioural marker systems assess non-technical skills in healthcare but suffer from variable rater training and poor inter-rater reliability.
  • Inter-rater reliability data reflects the tool's quality, not individual rater competence.
  • The pharmacists' behavioural skills (PhaBS) system is a new tool for assessing pharmacists' behavioural skills.

Purpose of the Study:

  • To test the inter-rater reliability of the PhaBS system using clinically experienced faculty and near-peer raters.
  • To assess rater competence using PhaBS, evaluating completeness, agreement with an expert, performance ranking, stringency/leniency, and halo effect avoidance.

Main Methods:

  • Clinically experienced faculty and near-peer raters received a 30-minute PhaBS familiarisation session.
  • Raters assessed trainee pharmacist behavioural skills in three simulated acute care scenarios (good, mediocre, poor performance).
  • Inter-rater reliability was calculated using intra-class correlation coefficient (ICC); rater competence differences were compared using Pearson's chi-squared test.

Main Results:

  • Experienced faculty achieved good inter-rater reliability (ICC=0.60), while near-peer raters showed poor reliability (ICC=0.38).
  • A higher proportion of experienced faculty raters (5/9) were competent across all domains compared to near-peer raters (2/13), though not statistically significant.
  • No significant differences were found between groups in agreement with an expert, performance ranking, stringency/leniency, or halo effect avoidance; however, experienced faculty were significantly better at completing the assessment (9/9 vs 6/13).

Conclusions:

  • Clinically experienced faculty demonstrate acceptable inter-rater reliability with PhaBS, aligning with other behavioural marker systems.
  • Not all raters achieve full competence, highlighting the need for rater competence frameworks in behavioural marker system assessments.
  • Implementing rater competence frameworks ensures fair assessments, informs rater training, and provides targeted feedback to individual raters.