Related Experiment Video
Updated: Jun 6, 2026

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
A review of critical appraisal tools show they lack rigor: Alternative tool structure is proposed
Michael Crowe1, Lorraine Sheppard
1Physiotherapy Department, James Cook University, Townsville Qld, Australia. michael.crowe@jcu.edu.au
Objectives:
To evaluate critical appraisal tools (CATs) that have been through a peer-reviewed development process with the aim of analyzing well-designed, documented, and researched CATs that could be used to develop a comprehensive CAT.
Study Design And Setting:
A critical review of the development of CATs was undertaken.
Results:
Of the 44 CATs reviewed, 25 (57%) were applicable to more than one research design, 11 (25%) to true experimental studies, and the remaining 8 (18%) to individual research designs. Comprehensive explanation of how a CAT was developed and guidelines to use the CAT were available in five (11%) instances. There was no validation process reported in 11 CATs (25%) and 33 CATs (77%) had not been reliability tested. The questions and statements that made up each CAT were coded into 8 categories and 22 items such that each item was distinct from every other.
Conclusions:
CATs are being developed while ignoring basic research techniques, the evidence available for design, and comprehensive validation and reliability testing. The basic structure for a comprehensive CAT is suggested that requires further study to verify its overall usefulness. Meanwhile, users of CATs should be careful about which CAT they use and how they use it.
Related Concept Videos
Lazarus's Cognitive Appraisal Theory
Primary Appraisal:...
Methods of Documentation III: PIE
Structuralism
Titchener's approach to structuralism was unique. He employed introspection, a method...
Reliability and Validity
Critical Thinking II
Decision Making: Traditional Method
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...