Related Experiment Video
Updated: Aug 24, 2026

Rup (RNA-seq Usability Assessment Pipeline) - Quality Control for Bulk RNA-seq Experiments in Eukaryotes
Published on: November 7, 2025
Reviewing measures of outcome: reliability of data extraction
K L Haywood1, J Hargreaves, R White
1Interdisciplinary Research Centre in Health, School of Health and Social Sciences, Whitefriars, Coventry University, UK.
Rationale, Aims And Objectives:
Although there is wide acceptance of the necessary domains for the evaluation of measures of health outcome, there has been little development or evaluation of methods of data extraction, such as checklists. This study aimed to assess the inter-rater reliability of data extraction from published articles using a new electronic database developed for structured reviews of health outcome measures.
Methods:
Domains considered important in the evaluation of measures of health outcome provided the foundation for a checklist for data extraction using a new electronic database. The database utilized Microsoft Access 2000 software (Microsoft Corporation, 2000). Following training, three independent reviewers extracted data from seven selected articles using the electronic database, and assessed article and measure quality against pre-defined criteria. Data extraction was compared for consistency and item content. Inter-reviewer agreement for categorical data was assessed by multiple kappa correlation.
Results:
Analysis demonstrated strong agreement between reviewers for all aspects of data extraction. However, lack of clarity in published articles affected the ability to clearly identify measures of outcome.
Conclusions:
The high level of inter-reviewer agreement supports the use of multiple trained reviewers in data extraction for reviews of measures of outcome using the checklist and Access software described. The electronic database supports standardized data extraction from published articles, benefiting from the combination of data extraction and data entry in a single step. Adaptation of the database to support structured reviews of measures of outcome adopted in the evaluation of other health states is proposed.
Related Concept Videos
Reliability and Validity
Statistical Analysis: Overview
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
Data Validation
Key parameters for method validation include:
Systematic Error: Methodological and Sampling Errors
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Variability: Analysis
The range is a simple measure of variability, indicating the difference between the highest and...
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
