Related Experiment Video
Updated: Aug 10, 2026

An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
Relative risk of a shuffled deck: a generalizable logical consistency criterion for sample selection in health state
Benjamin M Craig1, Sulabha Ramachandran
1University of Arizona College of Pharmacy, Tucson, AZ 85721, USA. craig@pharmacy.arizona.edu
Insights
Respondents in health state valuation studies can provide inconsistent rankings. A new statistical criterion identifies and removes "shuffled decks" to ensure accurate health valuation estimates.
Area of Science:
- Health Economics
- Biostatistics
- Psychometrics
Background:
- Health state valuation studies involve respondents ranking health states.
- Logical inconsistencies can arise, particularly with 'shuffled decks' of cards.
- Inconsistent responses can bias valuation estimates.
Purpose of the Study:
- To introduce a novel statistical criterion for identifying and excluding logically inconsistent responses in health state valuation.
- To ensure the integrity and accuracy of health state valuation estimates.
- To provide a generalizable method applicable across various health state classifications and valuation techniques.
Main Methods:
- Development of a logical consistency criterion based on the relative risk of a shuffled deck.
- Application of the criterion to secondary data from 4048 US and 3395 UK respondents.
- Utilized time trade-off and visual analog scale techniques for evaluating EQ-5D health states.
Main Results:
- A small proportion of respondents (approx. 5% in the UK) exhibited shuffled decks.
- Exclusion of these respondents significantly altered sample characteristics.
- Mean value estimates for EQ-5D health states were notably impacted by the exclusion.
Conclusions:
- The proposed logical consistency criterion is effective in identifying problematic respondents.
- Excluding respondents with shuffled decks is crucial for unbiased health state valuation.
- The criterion offers a robust method for improving the quality of health valuation data.
Abstract:
In a health state valuation study, respondents may be asked to rank a deck of cards, with each card representing a particular health state. A logical inconsistency occurs when a more severe health state card is ranked higher than a less severe card. Occasional inconsistencies may be justified by errors in judgment or measurement. However, when respondents return shuffled decks, their responses must be removed from the sample; otherwise, valuation estimates will be biased toward the median. In this paper, we present a logical consistency criterion for sample selection in health state valuation studies. This statistical criterion is based on the relative risk of a shuffled deck and generalizable to all health state classification systems, subsets (or decks) of health states, and valuation techniques. We applied the criterion to secondary data collected from 4048 United States and 3395 United Kingdom respondents. In both studies, respondents evaluated 12-card decks of EQ-5D health states using time trade-off and visual analog scale techniques. Among the UK respondents, a small portion (approximately 5%) did not satisfy the criterion; their exclusion significantly changed the sample characteristics and the mean value estimates of the EQ-5D health states. Similar results were found among the US respondents.
Related Concept Videos
Randomized Experiments
Simple randomization
Simple...
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast, controlled...
Relative Risk
Random Sampling Method
Bias
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
Testing a Claim about Population Proportion
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
