Exploring the potential of Artificial Intelligence in Traumatology: Conversational answers to specific questions

Generative Artificial Intelligence is a technology that provides greater connectivity with people through conversational bots («chatbots»). These bots can engage in dialogue using natural language indistinguishable from humans and are a potential source of information for patients.The aim of this study is to examine the performance of these bots in solving specific issues related to orthopedic surgery and traumatology using questions from the Spanish MIR exam between 2008 and 2023.

Three «chatbot» models (ChatGPT, Bard and Perplexity) were analyzed by answering 114 questions from the MIR. Their accuracy was compared, the readability of their responses was evaluated, and their dependence on logical reasoning and internal and external information was examined. The type of error was also evaluated in the failures.

ChatGPT obtained 72.81% correct answers, followed by Perplexity (67.54%) and Bard (60.53%).Bard provides the most readable and comprehensive responses. The responses demonstrated logical reasoning and the use of internal information from the question prompts. In 16 questions (14%), all 3 applications failed simultaneously. Errors were identified, including logical and information failures.

While conversational bots can be useful in resolving medical questions, caution is advised due to the possibility of errors. Currently, they should be considered as a developing tool, and human opinion should prevail over Generative Artificial Intelligence.