Related Experiment Video
Updated: Mar 7, 2026
Evaluating the Effectiveness of Cancer Drug Sensitization In Vitro and In Vivo
Published on: February 6, 2015
Validity and Reliability of Value Assessment Frameworks for New Cancer Drugs
Tanya G K Bentley1, Joshua T Cohen2, Elena B Elkin3
1Partnership for Health Analytic Research, LLC, Beverly Hills, CA, USA.
Background:
Several organizations have developed frameworks to systematically assess the value of new drugs. These organizations include the American Society of Clinical Oncology (ASCO), the European Society for Medical Oncology (ESMO), the Institute for Clinical and Economic Review (ICER), and the National Comprehensive Cancer Network (NCCN).
Objectives:
To understand the extent to which these four tools can facilitate value-based treatment decisions in oncology.
Methods:
In this pilot study, eight panelists conducted value assessments of five advanced lung cancer drugs using the ASCO, ESMO, and ICER frameworks. The panelists received instructions and published clinical data required to complete the assessments. Published NCCN framework scores were abstracted. The Kendall's W coefficient was used to measure convergent validity among the four frameworks. Intraclass correlation coefficients were used to measure inter-rater reliability among the ASCO, ESMO, and ICER frameworks. Sensitivity analyses were conducted.
Results:
Drugs were ranked similarly by the four frameworks, with Kendall's W of 0.703 (P = 0.006) across all the four frameworks. Pairwise, Kendall's W was the highest for ESMO-ICER (W = 0.974; P = 0.007) and ASCO-NCCN (W = 0.944; P = 0.022) and the lowest for ICER-NCCN (W = 0.647; P = 0.315) and ESMO-NCCN (W = 0.611; P = 0.360). Intraclass correlation coefficients (confidence interval [CI]) for the ASCO, ESMO, and ICER frameworks were 0.786 (95% CI 0.517-0.970), 0.804 (95% CI 0.545-0.973), and 0.281 (95% CI 0.055-0.799), respectively. When scores were rescaled to 0 to 100, the ICER framework provided the narrowest band of scores.
Conclusions:
The ASCO, ESMO, ICER, and NCCN frameworks demonstrated convergent validity, despite differences in conceptual approaches used. The ASCO inter-rater reliability was high, although potentially at the cost of user burden. The ICER inter-rater reliability was poor, possibly because of its failure to distinguish differential value among the sample of drugs tested. Refinements of all frameworks should continue on the basis of further testing and stakeholder feedback.
Insights
Four oncology drug value assessment frameworks showed similar drug rankings, indicating convergent validity. Refinements are recommended for these tools to improve value-based treatment decisions.
Area of Science:
- Health economics and outcomes research
- Oncology
- Health policy
Background:
- Multiple organizations have developed frameworks for systematic drug value assessment.
- Key organizations include the American Society of Clinical Oncology (ASCO), European Society for Medical Oncology (ESMO), Institute for Clinical and Economic Review (ICER), and National Comprehensive Cancer Network (NCCN).
Purpose of the Study:
- To evaluate the extent to which four major oncology drug value assessment frameworks facilitate value-based treatment decisions.
- To compare the convergent validity and inter-rater reliability of these frameworks.
Main Methods:
- Eight panelists assessed five advanced lung cancer drugs using ASCO, ESMO, and ICER frameworks.
- NCCN framework scores were abstracted from published data.
- Kendall's W coefficient measured convergent validity; intraclass correlation coefficients measured inter-rater reliability.
Main Results:
- All four frameworks demonstrated convergent validity, ranking drugs similarly (Kendall's W = 0.703, P = 0.006).
- Pairwise comparisons showed high agreement between ESMO-ICER and ASCO-NCCN.
- Inter-rater reliability varied, with ASCO and ESMO showing higher scores than ICER.
Conclusions:
- The ASCO, ESMO, ICER, and NCCN frameworks exhibit convergent validity for drug value assessment in oncology.
- While ASCO demonstrated high inter-rater reliability, ICER's was poor, potentially due to its scoring granularity.
- Ongoing refinement of these frameworks based on further testing and stakeholder input is crucial.
Related Concept Videos
01:25Mutagenicity and Carcinogenicity
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:21Cancer Survival Analysis
01:03Data Validation
Nursing assessment guides are generally based on holistic models rather than medical...
01:11Clinical Trials: Overview
01:10In Vitro Drug Release Testing: Overview, Development and Validation