Use of multiparameter evidence synthesis to assess the appropriateness of data and structure in decision models

Decision models for health technology appraisal are defined by their structure and data. Often there are alternatives for how the model might be specified and what data to include, and criteria are required to guide these choices. This study uses multiparameter evidence synthesis (MPES) to synthesize data from diverse sources and test alternative model structures. The methods are illustrated by a comparison of blood ketone testing versus urine ketone testing for young people with Type 1 diabetes.

Two approaches were compared. A simple statistical model (Model 1) was used to estimate the difference in the rates of adverse events from the outcome data of a randomized controlled trial (RCT). MPES (Model 2) was constructed to synthesize data on outcome and process variables from the RCT with data from nonrandomized studies on specificity and sensitivity. Sensitivity analyses were carried out using alternative model specifications for the MPES, and the consistency of the data was evaluated.

Model 1 estimated that the mean difference in the rate of adverse events per day was 0.0011 (95% confidence interval 0.0005-0.00229) lower with blood ketone testing. Model 2 estimated a similar outcome but also estimated parameters for which there were no direct data, including the prevalence of high ketone levels and the sensitivity and specificity of the tests as used in the home.

Model 1, which used only outcome data from an RCT, showed that blood ketone testing is more effective but did not explain why this is so. Model 2, estimated by MPES, suggested that the blood test is more accurate and that patients are more likely to comply with the protocol.