Related Experiment Video
Updated: Feb 5, 2026

An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
Reliability and Validity of Proposed Risk Stratification Methods for Laboratory Developed Tests
Jeffrey S Mohlman1, Jonathan R Genzen1, Ronald L Weiss1
1Department of Pathology, University of Utah and ARUP Laboratories, Salt Lake City, Utah.
Objectives:
To determine whether different laboratory developed test (LDT) risk stratification proposals would assign differing levels of risk to selected LDTs as a measure of the validity of those proposals, and whether there would be differing interrater agreement rates as a measure of the reliability of those proposals.
Methods:
A total of 4 reviewers applied 6 proposals for risk stratification of 4 LDTs. Interrater agreement was calculated as a measure of the reliability of the proposals. Also, a consensus risk categorization and concordance rate for each LDT was developed as a measure of the validity of the proposals.
Results:
Interrater agreement rates (reliability) ranged from 38% to 100%, and concordance rates (validity) ranged from 20% to 100%.
Conclusions:
A spectrum of reliability and validity was observed depending on the policy used and the LDT categorized. Before implementation or legislation of risk-stratification methods, large evaluations of reliability and validity should be conducted on any proposed method.
Related Concept Videos
Reliability and Validity
In Vitro Drug Release Testing: Overview, Development and Validation
Development of Analytical Methods
Relative Risk
Distribution Reliability and Automation
Data Validation
Key parameters for method validation include:

