Related Experiment Video
Updated: Jul 30, 2025

An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
Development of a Multi-Criteria Decision Analysis Rating Tool to Prioritize Real-World Evidence Questions for the
Ambica Parmar1,2, Wei Fang Dai2, Francois Dionne3
1Division of Medical Oncology & Hematology, Department of Medicine, Sunnybrook Health Sciences Centre, Toronto, ON M4N 3M3, Canada.
Abstract:
The Canadian Real-world Evidence for Value of Cancer Drugs (CanREValue) collaboration developed an MCDA rating tool to assess and prioritize potential post-market real-world evidence (RWE) questions/uncertainties emerging from public drug funding decisions in Canada. In collaboration with a group of multidisciplinary stakeholders from across Canada, the rating tool was developed following a three-step process: (1) selection of criteria to assess the importance and feasibility of an RWE question; (2) development of rating scales, application of weights and calculating aggregate scores; and (3) validation testing. An initial MCDA rating tool was developed, composed of seven criteria, divided into two groups. Group A criteria assess the importance of an RWE question by examining the (1) drug's perceived clinical benefit, (2) magnitude of uncertainty identified, and (3) relevance of the uncertainty to decision-makers. Group B criteria assess the feasibility of conducting an RWE analysis including the (1) feasibility of identifying a comparator, (2) ability to identify cases, (3) availability of comprehensive data, and (4) availability of necessary expertise and methodology. Future directions include partnering with the Canadian Agency for Drugs and Technology in Health's Provincial Advisory Group for further tool refinement and to gain insight into incorporating the tool into drug funding deliberations.
Insights
The Canadian Real-world Evidence for Value of Cancer Drugs collaboration created a tool to prioritize cancer drug evidence questions. This multi-criteria decision analysis (MCDA) tool helps assess the importance and feasibility of real-world evidence (RWE) studies.
Area of Science:
- Health Economics
- Pharmaceutical Policy
- Real-World Evidence
Background:
- Public drug funding decisions in Canada require robust evidence.
- Prioritizing post-market real-world evidence (RWE) questions is crucial for value assessment.
- Existing methods for prioritizing RWE questions are not standardized.
Purpose of the Study:
- To develop and validate a multi-criteria decision analysis (MCDA) rating tool.
- To assess and prioritize potential post-market RWE questions for cancer drugs.
- To support evidence-based drug funding decisions in Canada.
Main Methods:
- A three-step development process involving multidisciplinary stakeholders.
- Selection of criteria for assessing RWE question importance and feasibility.
- Development of rating scales, weighting, and aggregate score calculation, followed by validation testing.
Main Results:
- An initial MCDA rating tool with seven criteria was developed.
- Criteria assess RWE question importance (clinical benefit, uncertainty magnitude, decision-maker relevance).
- Criteria assess RWE feasibility (comparator identification, case ascertainment, data availability, expertise).
Conclusions:
- The MCDA tool provides a structured approach to prioritizing RWE questions.
- The tool can inform drug funding deliberations by focusing on high-priority evidence needs.
- Future work will involve refinement with the Canadian Agency for Drugs and Technology in Health.
Related Concept Videos
Decision Making: P-value Method
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can...
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Hazard Ratio
For example, in a clinical trial...
Decision Making: Traditional Method
First, a specific claim about the population parameter is decided based on the research question and is stated in a simple form. Further, an opposing statement to this claim is also stated. These statements can act as null and alternative hypotheses, out of which a null hypothesis would be a...
Sensitivity, Specificity, and Predicted Value
Sensitivity is the...
Response Surface Methodology
The process of RSM involves several key steps:

