A Scoping Review of Large Language Model Chatbot Use for Alcohol and Other Drug Health Information

While people prefer to seek alcohol and drug information (AOD) online, there can be quality and accessibility issues with these sources. Large Language Model (LLM) based chatbots are an emerging technology that may present an opportunity to overcome these barriers. We aimed to review the literature on the use of chatbots for seeking AOD health information, particularly the benefits, challenges and recommendations for future use.

Scoping review methodology was used to conduct a systematic search of four databases for English language studies relating to the use of chatbots to seek AOD health information in the last 5 years. This resulted in the screening of 243 articles, with five included studies.

There has been growing interest in the topic, though evidence is still limited. Despite identified benefits of chatbot use such as accuracy, appropriateness, overall experience and the provision of supporting documentation, important challenges in user safety concerns, lack of referral, quality, readability issues, and lack of adherence to current guidelines were noted, with mixed results regarding evidence-based responses. Only three of the five studies recommended chatbots for AOD-information seeking.

The current review suggests gaps in knowledge remain in the areas of accuracy, user safety, readability, evidence base and quality of LLM chatbot responses to AOD questions. More research is needed to investigate the applicability of LLMs in obtaining safe, non-stigmatising AOD information.