Risks of omission and commission errors when using generative AI in behavioral science research
Related Experiment Videos
Moritz Joerling1, Hauke Roggenkamp2, Geoff Tomaino3
1EM Lyon Business School, 15 Bd. Diderot, 75012, Paris, France.
Abstract:
Generative artificial intelligence (GenAI) is rapidly being adopted across the behavioral sciences, reshaping how researchers generate ideas, design studies, collect data, and analyze results. While GenAI offers substantial benefits such as increasing productivity, enabling more sophisticated methods, and lowering technical barriers, these gains come with potential costs in the form of errors. This article discusses the risks of errors as a foundation to decide when and how to use GenAI in behavioral research. We distinguish between two types of errors of shifting relative threat, depending on how narrowly a research task is specified. Open-ended tasks (e.g., ideation, literature review) leave a large space of plausible outputs and make researchers relatively more liable to errors of omission. In constrained, well-specified tasks (e.g., running a specified analysis, writing code to a stated scheme), the narrow solution space limits the scope for omission, raising the relative threat of errors of commission. We argue that commission errors are potentially easier to detect. Omission errors, by contrast, are harder to detect and pose greater risks to the diversity and scope of scientific inquiry. We propose that GenAI usage should be guided by a comparative assessment of error detectability. We offer practical recommendations for evaluating GenAI applications and for preserving the methodological diversity that sustains cumulative science.
Related Concept Videos
Systematic Error: Methodological and Sampling Errors
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Bias
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
Random and Systematic Errors
Random and Systematic Errors
Ethics in Research
Behavioral Genetics and Its Designs
The primary methodologies used in behavior genetics include family studies, twin studies, and adoption studies, each providing unique...