A novel experimental vignette methodology: SMART vignettes.

Area of Science:

• Decision Science
• Survey Methodology
• Health Informatics

Background:

• Traditional vignette studies offer limited causal inference and group balance.
• Sequential multiple assignment randomization trials (SMART) provide a foundation for advanced study designs.
• Understanding physician attitudes towards AI in clinical systems requires nuanced research methods.

Purpose of the Study:

• To propose and evaluate a novel vignette study design: SMART vignettes.
• To enhance causal inference, group allocation balance, and respondent interactivity in vignette studies.
• To demonstrate the utility of SMART vignettes in probing physician attitudes toward AI-embedded clinical systems.

Main Methods:

• Developed the SMART vignette design, incorporating sequential randomization and adaptive allocation.
• Applied SMART vignettes to a case study investigating physician attitudes toward AI in clinical systems.
• Simulated SMART and conventional vignette designs to compare group balance using relative loss.

Main Results:

• SMART vignettes demonstrated superior group balance compared to conventional designs across various sample sizes.
• Both designs showed improved balance with larger sample sizes.
• Simulations confirmed the advantage of the SMART vignette design in minimizing relative loss.

Conclusions:

• The proposed SMART vignette design offers significant advantages over traditional methods for causal inference and balance.
• SMART vignettes enhance respondent interactivity, providing richer data for decision-making research.
• This methodology holds promise for advancing knowledge in scenarios involving sequential decision-making.