Accuracy of large language model-based artificial intelligence tools for equine topics.

Area of Science:

• Equine Science
• Artificial Intelligence Applications

Background:

• Artificial intelligence (AI) platforms are increasingly utilized for equine information retrieval.
• These platforms synthesize data from diverse sources, often without differentiating between factual information and opinion.

Purpose of the Study:

• To evaluate the accuracy and quality of AI-generated responses to equine-related queries.
• To test the hypothesis that AI platforms excel at basic equine questions but falter on complex subjects.

Main Methods:

• Forty equine-related questions were developed across five categories: general care, facilities, nutrition, genetics, and reproduction.
• Questions were classified by difficulty (beginner, intermediate, advanced, trending) and posed to three AI platforms: ChatGPT, Microsoft Copilot, and ExtensionBot.
• Responses were scored on accuracy, relevance, thoroughness, and source quality (maximum 20 points each).

Main Results:

• AI platform performance varied significantly by question difficulty, topic, and platform.
• ChatGPT and Microsoft Copilot demonstrated higher accuracy and relevance compared to ExtensionBot.
• Intermediate-level questions received the highest overall scores, while complex topics presented challenges for AI accuracy and thoroughness.

Conclusions:

• AI platforms can serve as supplementary resources for equine information but do not currently match the expertise of Equine Extension Specialists.
• AI performance is inconsistent, particularly when addressing intricate equine subjects, highlighting areas for improvement.