Qualitative and Quantitative Analysis of Interviewer Help Answering the Time Tradeoff

The time tradeoff (TTO) is an important method to directly obtain health utilities. Challenges of the TTO are, among others, "nontraders" and illogical answers. In TTO interviews, these challenges are resolved by the interviewer. In web-based TTOs, training procedures and logical checks are used based on the views of the researchers. As web-based TTOs will be used more often in the future, we investigated how respondents arrive at their ratings to determine the help they require.

In 2 earlier studies performed by this research group, respondents valued 6 EQ-5D states on a TTO. Respondents were asked to think out loud, and all interviews were audiotaped. A random selection of these interviews were transcribed and double-coded by two independent raters, using a priori and inductive coding until saturation was reached. Based on the retrieved mistakes and comments, a list of frequently asked questions (FAQ) was developed.

In total, 91 interviews were coded. In all, 85% made at least 1 mistake, 41% made a misreading/miscalculation, 70% misunderstood the tradeoff, 27% misunderstood the EQ-5D dimensions, 29% misunderstood the scenario, 45% made a comment about the TTO, and 43% expressed frustration. More misunderstandings were reported in the Peeters study, which was performed in a realistic setting, whereas the van Osch study was conducted in a more ideal setting. Misunderstandings of the scenario were mosly reported by patients.

Almost all respondents need interviewer help. This may have implications for the validity of interviewer-based TTO elicitations when social acceptability bias is an issue or with explicit hypothesis and the interviewer is not blinded. The FAQ list can be used to standardize interviewer help or as a help function in a web-based TTO.