Can Algorithms Efficiently Identify Interpretable and Persuasive Message Features? An Agnostic Causal Machine...

Argument strength and message persuasiveness are key constructs in message effects research. Yet, researchers still lack a systematic and efficient approach to uncover the "recipe" for these message-level latent features. We applied an agnostic causal machine learning approach that integrates the supervised Indian Buffet Process (sIBP) algorithm with AI-facilitated researcher refinement, train/test set splitting, and crowdsourcing in two multiple-message experiments, each with a large stimulus pool. We conducted message-level analyses on (a) textual tobacco control messages (K = 377) among Chinese men who smoke (N = 1,206) and (b) COVID-19 vaccine promotional messages (K = 1,759) among a national sample of U.S. adults (N = 819). This agnostic approach discovered that messages emphasizing negative health consequences increased argument strength and message persuasiveness. In contrast, politicizing cues reduced the message persuasiveness of social media messages promoting COVID-19 vaccines. We discussed the strengths and limitations of this approach for future message effects research.