Multimodal Analysis of Child-Horse Therapeutic Interactions Using Vision-Language Models in an Australian Aboriginal Equine-Assisted Learning Program for Social and Emotional Well-Being: Development and Evaluation Study

The human-horse bond has played a role in human development throughout history and has more recently been applied to mental health promotion through equine-assisted learning (EAL) programs. In EAL settings, therapeutic documentation and assessment commonly rely on practitioner observations documented through behavioral checklists, structured interviews, and progress notes, which can pose challenges due to time demands, subjectivity, and inconsistent standardization across practitioners and contexts.

This study aimed to address these challenges in the context of mental health promotion programs. This study developed an AI-powered digital health framework based on vision-language models (VLMs) to analyze child-horse therapeutic interactions in EAL settings. The framework was developed in collaboration with the Yawardani Jan-ga EAL program, which supports the social and emotional well-being of Aboriginal youth in the Kimberley region of Western Australia.

The proposed framework comprises three core components: (1) an activity classification module using a VLM with few-shot learning based on Sigmoid Loss for Language-Image Pre-training 2 (SigLIP2) to identify 9 therapeutic activities; (2) a proxemics analysis module using a Qwen2.5-VL-7B VLM to classify physical contact between the child and horse; and (3) a body posture analysis module applying multimodal chain-of-thought prompting and self-consistency techniques to classify children's body posture into confidence-related states with interpretable reasoning.

The framework was developed and evaluated using real-world images collected during therapeutic sessions with Aboriginal children participating in the Yawardani Jan-ga program. The activity classification module achieved an accuracy of 90.6%, with macroaveraged precision, recall, and F₁-scores of 92.0%, 91.1%, and 91.1%, respectively, under the selected 2-shot configuration. The proxemics analysis module achieved an accuracy of 88.6%, with macroaveraged precision, recall, and F₁-scores of 89.0%, 88.0%, and 88.5%, respectively, under the 1-shot configuration. For the body posture analysis module, system reliability was assessed using internal consistency metrics, achieving an average consistency of 0.916, an agreement rate of 0.705, a rationale coherence of 0.857, and a vote entropy of 0.174.

Overall, the findings suggest that AI- and VLM-based approaches can provide complementary support for practitioner-led documentation and assessment of observable child-horse therapeutic interactions in EAL settings. These approaches may help reduce practitioner burden and strengthen the assessment of programs that promote children's social and emotional well-being.