Can Consumers Rely on AI Chatbots for Food Safety Advice? A Comparative Analysis of Chatbot Responses and Food Safety...

The growing integration of artificial intelligence tools, particularly conversational agents, is transforming how consumers access information, including content related to critical areas such as public health and personal healthcare. A portion of the American population already uses chatbots for health-related guidance, and many express trust in their information. The objectives of this study were to assess chatbot-generated content on consumer food safety and compare it with specialists' guidance. In Study 1, a comparative analysis was conducted to assess the food safety risk evaluation performance of four popular chatbots available to consumers: ChatGPT-3.5, Google Bard, Meta AI, and Microsoft Copilot. Twenty food handling practices discussed in the podcast Risky or Not were selected. For each practice, a standardized Prompt 1 was inserted "Is there a food safety risk?". In Study 2, we compared three different ChatGPT versions (-3.5, -4.o mini, and -5.2). Study 3 investigated whether slight variations using Prompt 2 and 3 affected the alignment rate of ChatGPT-4.o mini with food safety experts. Study 4 evaluated whether ChatGPT-4.o mini could provide evidence to support its statements with valid references. To assess the response consistency, each question was issued to the chatbot three times for all studies. Results revealed substantial variability in chatbot performance, with Microsoft Copilot aligned more closely with food safety specialists than Google Bard, Meta AI, and ChatGPT-3.5, respectively. The prompt variations on ChatGPT-4.o mini indicated that how consumer phrases in inquiries might affect chatbot responses, and its performance increased compared to the first evaluation. However, our results suggest that prompt design alone was insufficient to ensure the chatbot's reliability as an exclusive source of food safety advice to consumers. The limited alignment with expert recommendations, inconsistency across repetitions, and overly assertive tone of chatbots, which might mislead consumers, highlight the need for additional evidence-based evaluation and open communication with consumers to support informed consumer decision-making in using chatbots for food safety risk assessment.