Related Experiment Video
Updated: Jun 29, 2026
10:39
Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
Assessing the quality of diagnostic studies using psychometric instruments: applying QUADAS
Rachel Mann1, Catherine E Hewitt, Simon M Gilbody
1Dept. of Health Sciences, University of York, York YO10 5DD, UK.
Social Psychiatry and Psychiatric Epidemiology
|October 7, 2008
Summary
The Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool shows good agreement among reviewers for evaluating diagnostic accuracy studies with psychometric instruments. However, poor reporting quality in studies hinders its full application.
Area of Science:
- Medical research methodology
- Diagnostic test evaluation
- Psychometrics
Background:
- Increasing systematic reviews of diagnostic tests necessitate standardized reporting and quality assessment tools.
- The STARD (STatement for Reporting of Diagnostic Accuracy) and QUADAS (Quality Assessment of Diagnostic Accuracy Studies) checklists were developed in response.
Purpose of the Study:
- To assess the validity and utility of the QUADAS checklist for diagnostic accuracy studies employing psychometric instruments.
- To evaluate the quality of reporting in these studies during the practical application of the QUADAS checklist.
Main Methods:
- Two independent reviewers assessed the quality of 54 diagnostic accuracy studies using the QUADAS checklist.
- Inter-rater agreement was calculated using the proportion of agreement for overall and item-specific assessments.
Main Results:
- Overall agreement between reviewers for QUADAS items was 85.7%.
- Agreement exceeded 80% for 8 items, but was poorest for selection criteria, indeterminate results, and withdrawals.
- No studies provided adequate reporting for all QUADAS items to be scored as 'yes'.
Conclusions:
- The QUADAS checklist is a user-friendly and acceptable tool for appraising diagnostic accuracy studies using psychometric instruments.
- The practical application of QUADAS is significantly limited by the prevalent poor quality of study reporting.
Related Concept Videos
01:17Cochran's Q Test
Cochran's Q Test is a nonparametric statistical test used to determine if there are potential differences in the outcomes of three or more related groups on a binary (yes/no) or dichotomous outcome. It is essentially an extension of the McNemar Test, which is limited to two related samples - Cochran's Q test can handle three or more related samples, making it more versatile in scenarios where subjects are measured under multiple conditions. The test statistic follows a Chi-Square distribution,...
01:29Measures of Intelligence
Psychologists measure intelligence by using standardized tests that produce a score known as the intelligence quotient or IQ. To understand IQ tests, it's important to recognize the key principles behind their construction: validity, reliability, and standardization.
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this; it...
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this; it...
01:19Quality Assurance
Quality assurance is the overarching term used to describe the activities employed to ensure the proper performance of a system. These activities can be classified into three categories: quality control, quality assessment, and internal corrective measures. Typically, these activities work cyclically: quality control is performed before and during the analysis, while quality assessment occurs during and after the investigation. Internal corrective measures are implemented based on the findings...
01:29Reliability and Validity
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
01:22Self-Report Tests of Personality
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
01:13Sensitivity, Specificity, and Predicted Value
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...