The Evaluation of Decision Support Tools Requires a Measure of Decision Quality That Has Content and Construct...

The evaluation of decision aids and support interventions requires a multi-attribute index which can be calculated for both aid and comparator/s. The Decision Conflict Scale (DCS) is such an index and has been widely used in this context, the recent Cochrane review Decision aids for people facing health treatment or screening decisions reporting its use in 63 of the 105 studies included. However, while the DCS may be a valid measure for the eponymous construct - decision conflict - it lacks both content and construct validity for the evaluation of decision aids. It lacks content validity for this task because of the 3 items which make up its Uncertainty subscale, which penalize an aid that correctly reports the situation is one of decisional equipoise or near equipoise. A 'false clarity' bias in aid presentation is encouraged by being rewarded. In this paper we confirm that the inclusion of the Uncertainty subscale in the DCS has inappropriate empirical consequences for decision aid evaluation. Excluding the Uncertainty items would address this content invalidity, but the DCS would still lack construct validity, since the construct being measured is being treated, and inappropriately psychometrically validated, as a reflective rather than formative one. The component scales (items) of an index for a formative construct need to be preference-weighted, in ethical person-centred care by the individual at the point of decision. We argue that the most appropriate formative construct for use in decision aid evaluation is decision quality, and suggest MyDecisionQuality (MDQ) as the first formative index that can claim to both possess content and construct validity for the measurement of decision quality. However, like all multi-criterial formative metrics, the construct that MDQ measures is constructed by the measure and has no existence independent of it.