Usability of the "Be Yourself" IA-powered, chatbot-based website for the prevention of adolescent pregnancy

Objective. Analyze the usability, interaction patterns, and quality of conversational interaction of the AI-powered "Be Yourself" chatbot to foster self-determined motivation and prevent adolescent pregnancy.

Post-intervention usability pilot study conducted with 74 Mexican school-going adolescents (aged 11-15), using the System Usability Scale (SUS) at two time points: immediately after the intervention and at two-month follow-up. Interaction metrics (number of messages) and confidence coefficients were analyzed within content related to self-determined motivation, pregnancy prevention, and sexual communication.

Participants were 49.6% male and 50.4% female, with a mean age of 12.7 (SD=0.98). The chatbot recorded 74 chats and 1,236 messages. The SUS was assessed post-intervention (n=74) and at the two-month follow-up (n=70). Initially, usability was rated as 18.9% "Good" and 81.1% "Acceptable." At follow-up, the proportions across categories changed to 5.7% "Good," 92.9% "Acceptable," and 1.4% "Poor." The Wilcoxon signed-rank test showed a significant decrease in perceived usability (p<0.001; r=-0.56) between the two assessment points. The highest confidence coefficients were observed for communication with parents (93.2%) and pregnancy prevention (89.1%), whereas self-determined motivation was lower (79.1%).

The chatbot demonstrated acceptable usability and high short-term acceptance as an informational tool for preventing adolescent pregnancy. However, the decrease in usability scores at the two-month follow-up indicates the need for improvements to foster sustained engagement and address behavioral aspects.