Related Experiment Video
Updated: May 27, 2026

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
Effect of rating scales on scores given to junior doctors in multi-source feedback
Andrew Hassell1, Alison Bullock, Andrew Whitehouse
1Postgraduate Medical and Dental Education, NHS West Midlands Workforce Deanery, Birmingham, UK. a.hassell@keele.ac.uk
Background:
Multi-source feedback (MSF) has an established role in the workplace based assessment of doctors in training. Different models of MSF are currently used in different training programmes and settings. One important way in which these models differ is the rating scale on which assessors score the trainee. The aim of this study was to explore the effect of rating scale on MSF scores.
Methods:
Foundation Year 2 trainees in hospitals in the West Midlands underwent MSF using the validated MSF tool, team assessment of behaviour (TAB) in autumn 2005. Trainees were scored with TAB using one of four different rating scales, ranging from 3- to 9-point scales. Each participating hospital used only one rating scale. The proportions of trainees scored as having potential problems were related to the different rating scale used. Similarly, the proportions scored as 'above expectations' were compared. Assessors also completed a short questionnaire regarding the assessment.
Results:
245 trainees underwent 2594 assessments. Longer rating scales were associated with a lower proportion of trainees awarded 'problem' scores and higher proportions of trainees scored as 'above expectations'. Assessors generally reported no difficulties whichever rating scale they had used.
Conclusion:
Careful consideration, recognising its potential impact on assessment score, should be given to the rating scale used when instituting MSF within a system of workplace based assessment.
Related Concept Videos
Obedience
Ratio Level of Measurement
A set of data measured using the ratio scale takes care of the ratio problem and provides complete information. Ratio scale data are like interval scale data, except they have a zero point and ratios can be calculated. For...
Sources of Self-Esteem II: Performance Feedback
Surveys
Regression Toward the Mean
Friedman Two-way Analysis of Variance by Ranks