Related Experiment Video
Updated: Jun 8, 2026

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
Using standardized essays in the veterinary medicine admissions process: are the ratings reliable and valid?
1Health Sciences Centre, Department of Veterinary Clinical and Diagnostic Sciece, Faculty of Veterinary Medicine, University of Calgary, Calgary, AB, Canada. kghecker@ucalgary.ca
Abstract:
The reliability and validity of using essays for veterinary medical school admissions requires investigation. We explored the reliability and construct validity of a structured essay station in the 2009 admission process at the University of Calgary Faculty of Veterinary Medicine. One hundred three applicants (female=80.6%, male=19.4%; mean age=23.05 years, SD=3.96) participated. Each applicant wrote a one-hour supervised essay (750 words). Essays were rated independently by two randomly assigned raters (n=16). Raters scored essays on four items, each on a five-point anchored scale. Nine essays were scored by all raters to perform a decision study. Generalizability analysis resulted in a reliability coefficient of 0.55. The decision study indicated that three raters and four items produces a G of 0.68. Essay score correlated with interview score (r=0.30, p<0.01) but not with GPA (r=0.05, p=ns). Overall reliability was adequate and higher than what has been reported for unsupervised written submissions. Results from the decision study suggest that three raters with four items provide adequate reliability. Correlations with interviews and grade point average provide evidence of construct validity. A time-limited essay with a clear scoring protocol results in adequate reliability and some validity.
Related Concept Videos
Reliability and Validity
Surveys
Ethics in Research
Review and Preview
Percentiles are a type of fractile that partition data into...
Measures of Intelligence
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this; it...
Toxicity Testing in Animals
