Related Experiment Video

Updated: May 4, 2026

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
10:26

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities

Published on: September 11, 2021

3.1K

Reliability of a minimal competency score for an annual skills mastery assessment

Gregory L Alston1, Wesley R Haltom1

  • 1School of Pharmacy, Wingate University, Wingate, North Carolina.

American Journal of Pharmaceutical Education
|December 28, 2013
PubMed
Abstract

Insights

The modified Angoff process reliably establishes minimal competency scores for the Annual Skills Mastery Assessment (ASMA). Faculty panels accurately judged item difficulty and determined consistent cut scores across rating sessions.

Area of Science:

  • Medical Education
  • Psychometrics
  • Pharmacy Education

Background:

  • Establishing reliable minimal competency scores is crucial for standardized assessments in professional education.
  • The modified Angoff method is a widely used technique for setting performance standards on tests.

Purpose of the Study:

  • To evaluate the reliability and validity of the modified Angoff process for determining a minimal competency score on the Annual Skills Mastery Assessment (ASMA).

Main Methods:

  • Three panels of pharmacy faculty utilized the modified Angoff method to establish a minimal competency score for 60 existing ASMA test items.
  • Data analysis focused on inter-session variability, faculty type, item difficulty, and potential rater bias.

Main Results:

  • The determined cut scores showed no significant differences based on rating session or faculty type.
  • The range of cut scores across the examination was less than 3%.
  • Faculty panelists demonstrated accuracy in predicting student performance on items categorized as easy, medium, and hard.

Conclusions:

  • A well-structured faculty panel can reliably determine a minimal competency cut score using the modified Angoff process.
  • The modified Angoff process allows for accurate ranking of test item difficulty.
Keywords:
Angoff methodassessmentcompetencymile marker examinationmodified Angoff methodprogress examination

More Related Videos

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity08:40

Isokinetic Robotic Device to Improve Test-Retest and Inter-Rater Reliability for Stretch Reflex Measurements in Stroke Patients with Spasticity

Published on: June 12, 2019

6.7K
Development of a Virtual Reality Assessment of Everyday Living Skills10:32

Development of a Virtual Reality Assessment of Everyday Living Skills

Published on: April 23, 2014

20.2K

Related Concept Videos

Self-Evaluation Maintenance Model01:29

Self-Evaluation Maintenance Model

The Self-Evaluation Maintenance (SEM) model offers a psychological framework to understand how individuals’ self-esteem is influenced by the achievements of others, particularly those with whom they share close personal bonds. The SEM model operates when personal rather than social identity guides individuals. Central to this model is the notion that individuals have an inherent desire to preserve a favorable self-image, which is continuously shaped by interpersonal comparisons and...
423
Reliability and Validity01:29

Reliability and Validity

Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.9K
Spearman's Rank Correlation Test01:20

Spearman's Rank Correlation Test

Spearman's rank correlation test, also known as Spearman's rho, is a nonparametric method for assessing the strength and direction of association between two variables. This test is particularly valuable when the data distribution is unknown or when the assumption of normality does not hold. Named after the English psychologist and statistician Dr. Charles Edward Spearman, it serves as the nonparametric counterpart to Pearson's correlation coefficient.
Spearman's test calculates correlation by...
1.3K
Confidence Coefficient01:24

Confidence Coefficient

The confidence coefficient is also known as the confidence level or degree of confidence. It is the percent expression for the probability, 1-α, that the confidence interval contains the true population parameter assuming that the confidence interval is obtained after sufficient unbiased sampling; for example, if the CL = 90%, then in 90 out of 100 samples the interval estimate will enclose the true population parameter. Here α is the area under the curve, distributed equally under...
9.2K
Self-Report Tests of Personality01:22

Self-Report Tests of Personality

Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
1.3K
Uncertainty in Measurement: Accuracy and Precision03:37

Uncertainty in Measurement: Accuracy and Precision

Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. 
93.9K