Related Experiment Video
Updated: Jan 11, 2026
07:31
Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
7.5K
Developing a framework for assessing the applicability of the target condition in diagnostic research
Eve Tomlinson1, Jude Holmes2, Anne W S Rutjes3
1Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK.
Journal of Clinical Epidemiology
|November 17, 2025
Summary
Assessing study applicability in diagnostic test accuracy (DTA) reviews is challenging. We developed a framework to guide authors in evaluating the QUADAS-2 reference standard domain, improving the reliability of DTA reviews.
Area of Science:
- Medical research methodology
- Evidence synthesis
- Diagnostic accuracy studies
Background:
- Assessing the applicability of primary studies is crucial but difficult in systematic reviews of diagnostic test accuracy (DTA) studies.
- The QUADAS-2 tool is used to evaluate the risk of bias and applicability concerns in DTA reviews.
- Authors often struggle with accurately assessing and reporting applicability concerns, particularly regarding the reference standard domain.
Purpose of the Study:
- To explore how review authors assess applicability for the QUADAS-2 reference standard domain in Cochrane DTA reviews.
- To identify common applicability concerns and potential issues in the assessment process.
- To develop a framework to guide authors in assessing the applicability of the target condition as defined by the reference standard.
Main Methods:
- Methodological review of Cochrane DTA reviews that used QUADAS-2 and identified "high concern" for the reference standard domain.
- Extraction and thematic categorization of the rationale for "high concern" applicability judgments by two independent reviewers.
- Development of an applicability assessment framework based on extracted themes and expert discussion.
Main Results:
- Fifty eligible reviews were analyzed, revealing five main themes of applicability concerns.
- Key concerns included differing reference standard thresholds, misclassification of the target condition, and insufficient information provided.
- A framework was developed outlining four potential applicability concerns: target condition sub-categories, threshold differences, partial application of the reference standard, and misclassification.
Conclusions:
- While sources of applicability concerns are identifiable, many review authors face challenges in their assessment and reporting.
- The developed framework provides guidance for authors to improve the assessment of applicability concerns within the QUADAS reference standard domain.
- Enhanced applicability assessments contribute to more reliable and trustworthy DTA reviews.
Related Concept Videos
01:17Cochran's Q Test
930
Cochran's Q Test is a nonparametric statistical test used to determine if there are potential differences in the outcomes of three or more related groups on a binary (yes/no) or dichotomous outcome. It is essentially an extension of the McNemar Test, which is limited to two related samples - Cochran's Q test can handle three or more related samples, making it more versatile in scenarios where subjects are measured under multiple conditions. The test statistic follows a Chi-Square...
930
01:15Receiver Operating Characteristic Plot
464
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
464
01:28Criteria for Causality: Bradford Hill Criteria - II
1.1K
The Bradford Hill criteria serve as guidelines for establishing causative links in epidemiological research. Beyond Strength, Consistency, Specificity, and Temporality, key criteria also include Biological Gradient, Plausibility, Coherence, Experiment, and Analogy. These principles assist scientists in assessing the likelihood of causation in complex biological contexts. Below is a summary of these concepts:
1.1K
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
387
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
387
01:25Formulating and Validating Nursing Diagnosis II
3.6K
Nursing diagnoses represent a problem validated by major defining characteristics. There are four categories of nursing diagnoses: problem-focused, risk, health promotion or wellness, and syndrome. The anatomy of a nursing diagnosis includes three components: problem statement or diagnostic label, defining characteristics, and related factors.
Risk nursing diagnoses represent clinical judgments of an individual, family, or community more vulnerable to developing the health problem than others...
Risk nursing diagnoses represent clinical judgments of an individual, family, or community more vulnerable to developing the health problem than others...
3.6K
01:29Reliability and Validity
13.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
13.7K