An AI-Assisted Adaptive Boolean Rubric for exercise prescription evaluation: A pilot validation study

Area of Science:

• Exercise Science
• Artificial Intelligence in Healthcare
• Clinical Evaluation Tools

Background:

• Traditional rating scales for personalized exercise prescriptions lack objectivity and efficiency.
• Artificial intelligence (AI) offers a path toward scalable and objective quality assessment.

Purpose of the Study:

• To develop an AI-assisted evaluation framework for personalized exercise prescriptions.
• To assess the feasibility, reliability, and efficiency of the Adaptive Precision Boolean Rubric (Adaptive-PBR).

Main Methods:

• Developed a 50-item Precision Boolean Rubric (PBR) and a 10-item Likert scale based on ACSM guidelines.
• Utilized GPT-4 to generate 20-item Adaptive-PBRs for five diverse clinical cases.
• Twelve experts evaluated cases using PBR, Likert scale, and Adaptive-PBR under randomized conditions.

Main Results:

• Adaptive-PBR showed excellent inter-rater reliability (ICC = 0.83), outperforming the Likert scale (ICC = 0.65).
• Evaluation time was reduced by approximately 63% compared to the full PBR (7.1 vs. 19.5 min).
• Adaptive-PBR mitigated subjective variability and bias inherent in the Likert scale.

Conclusions:

• The AI-assisted Adaptive-PBR provides a feasible, reliable, and efficient standard for evaluating exercise prescriptions.
• This tool combines granular criteria with AI adaptability, offering a robust alternative to subjective scales.
• The Adaptive-PBR has immediate potential as a quality assurance tool in clinical settings.