Cross-model consistency of AI-generated exercise prescriptions: A repeated generation study across three large...

Area of Science:

• Artificial Intelligence
• Exercise Science
• Clinical Applications

Background:

• Large language models (LLMs) are increasingly used for exercise prescription.
• Cross-model consistency in LLM outputs for exercise prescription is unexamined.

Purpose of the Study:

• Systematically compare repeated generation consistency of exercise prescription outputs across three leading LLMs.
• Evaluate differences in semantic similarity, reproducibility, FITT component classification, and safety expression.

Main Methods:

• GPT-4.1, Claude Sonnet 4.6, and Gemini 2.5 Flash generated exercise prescriptions for six clinical scenarios, each 20 times (360 total outputs).
• Analysis included semantic similarity (SBERT cosine similarity), output reproducibility, FITT component classification, and safety expression under controlled conditions (temperature=0).

Main Results:

• GPT-4.1 exhibited the highest semantic similarity (0.955) with 100% unique outputs.
• Gemini 2.5 Flash showed high similarity (0.950) but low uniqueness (27.5%), indicating repetition over consistent reasoning.
• Claude Sonnet 4.6 had lower similarity (0.903); safety expression was high across all models, limiting its differentiation value.

Conclusions:

• Model-specific output behavior is crucial for evaluating LLMs in exercise prescription.
• Applications requiring reproducible and guideline-compatible exercise outputs necessitate careful consideration of LLM generation consistency.