Quality-checking the new normal: trial modality in online jury decision-making research

We sought to examine differences between videotaped and written trial materials on verdicts, perceptions of trial parties, quality check outcomes, perceived salience of racial issues, and emotional states in a trial involving a Black or White defendant.

We predicted that verdicts and ratings of trial parties would be similar for those participants viewing a videotaped trial and those reading a written transcript. However, we suspected that emotional states might be heightened for those watching a video and that those reading transcripts would perform better on quality checks regarding trial content (but worse on those involving trial party characteristics, including defendant race).

Participants (N = 139 after removing those who did not meet our threshold for data quality) recruited from Amazon's Mechanical Turk were randomly assigned to watch a video or read a transcript of a trial involving an alleged murder of a police officer. They completed a questionnaire probing their verdict, perceptions of trial parties, perceived salience of racial issues, and emotional state, and responded to a series of quality checks.

Participants in the videotape condition performed significantly worse on quality checks than did those in the transcript condition. There were no significant differences between modalities in terms of verdict or perceived salience of racial issues. Some other differences emerged between conditions, however, with more positive perceptions of the pathologist and police officer in the transcript condition, and more negative emotion elicited by the trial involving a White defendant in the videotape condition only.

There were no meaningful differences between videotaped and written trial materials in terms of outcome (verdict), but the presence of some trial party rating and emotional state differences stemming from modality epitomizes the internal/ecological validity trade-off in jury research. Our quality check results indicate that written transcripts may work better for obtaining valid data online. Regardless of modality, researchers must be diligent in crafting quality checks to ensure that participants are attending to the stimulus materials, particularly as more research shifts online.