Artificial intelligence generated clinical score sheets: looking at the two faces of Janus

Area of Science:

• Preclinical Research
• Animal Welfare Science
• Artificial Intelligence in Research

Background:

• Clinical score sheets are vital for objective animal welfare assessment in vivo.
• Artificial Intelligence (AI) is transforming preclinical research documentation.
• Standardized score sheets are crucial for minimizing animal distress.

Purpose of the Study:

• To evaluate Large Language Models (LLMs) for generating animal welfare score sheets.
• To assess the performance of ChatGPT-4, ChatGPT-3.5, and Google Bard in creating score sheets for an inflammatory bowel disease mouse model.
• To analyze LLM-generated score sheets for structural consistency, accuracy, and intervention appropriateness.

Main Methods:

• LLMs were prompted to generate clinical score sheets based on predefined criteria (weight loss, stool consistency, fecal blood).
• Three LLMs (ChatGPT-4, ChatGPT-3.5, Google Bard) were evaluated.
• Key parameters assessed included output structure, severity representation, and intervention thresholds.

Main Results:

• LLMs demonstrated consistent structural output for score sheets.
• Significant variations were observed in numerical symptom scoring and intervention threshold definition across models.
• AI shows potential for drafting but requires human expert validation.

Conclusions:

• LLMs can assist in creating foundational drafts of animal welfare score sheets.
• Expert oversight is critical for ensuring the accuracy and reliability of AI-generated preclinical research documentation.
• Balancing AI tools with professional judgment is essential for robust animal welfare assessment.