Related Experiment Video
Updated: Mar 23, 2026

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
When Assessment Data Are Words: Validity Evidence for Qualitative Educational Assessments
David A Cook1, Ayelet Kuper, Rose Hatala
1D.A. Cook is professor of medicine and medical education, associate director, Mayo Clinic Online Learning, and consultant, Division of General Internal Medicine, Mayo Clinic College of Medicine, Rochester, Minnesota.A. Kuper is assistant professor, Department of Medicine, Faculty of Medicine, University of Toronto, scientist, Wilson Centre for Research in Education, University Health Network/University of Toronto, and staff physician, Division of General Internal Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.R. Hatala is associate professor of medicine and director, Clinical Educator Fellowship, University of British Columbia, Vancouver, British Columbia, Canada.S. Ginsburg is professor, Department of Medicine, Faculty of Medicine, University of Toronto, scientist, Wilson Centre for Research in Education, University Health Network/University of Toronto, and staff physician, Mount Sinai Hospital, Toronto, Ontario, Canada.
Qualitative assessments offer richer insights into learner performance than quantitative scores alone. This study proposes rigorous standards for qualitative assessment validity, enhancing judgment defensibility in health professions education.
Area of Science:
- Health Professions Education
- Assessment Science
- Qualitative Research Methods
Background:
- Quantitative scores inadequately capture all aspects of learner performance.
- Increased reliance on qualitative data in health professional assessment.
- Challenges exist in understanding the role and rigor of qualitative assessments.
Purpose of the Study:
- To articulate the role of qualitative assessment within comprehensive assessment programs.
- To translate validity concepts for judgments derived from qualitative assessments.
- To propose standards for appraising the rigor of qualitative assessments.
Main Methods:
- Identified standards for rigor in qualitative research (responsiveness, reflexivity, sampling, description, triangulation, transparency, transferability).
- Reframed these standards using contemporary validity frameworks (Messick's and Kane's).
- Illustrated the approach with published research on learning portfolios.
Main Results:
- Established a framework for collecting and evaluating validity evidence for qualitative assessments.
- Demonstrated how to apply Messick's and Kane's validity concepts to qualitative data.
- Provided a structured approach to enhance the defensibility of qualitative assessment judgments.
Conclusions:
- Advocates for a "methods-neutral" approach to assessment, prioritizing purpose.
- Increased use of qualitative assessments requires more rigorous validity judgments.
- Strategic evidence collection is crucial for coherent validity arguments in assessment.
More Related Videos
10:26Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
12:55Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties
Published on: September 27, 2020
Related Concept Videos
Reliability and Validity
Data Validation
Nursing assessment guides are generally based on holistic models rather than medical...
Data Validation
Key parameters for method validation include:
Qualitative Analysis
For instance, group IV...
Qualitative Analysis
There are two main approaches to qualitative analysis:...
Case Studies