Usability and Workflow Integration of a Machine Learning-Derived Neonatal Risk Predictor in Kenyan Neonatal Units:...

Neonatal mortality remains a leading contributor to under-5 deaths globally, particularly in low- and middle-income countries (LMICs). While machine learning (ML)-based risk prediction models show promise for identifying high-risk neonates, published evidence describing the real-world usability and practical implementation of ML-derived neonatal risk prediction tools within routine clinical workflows in LMIC neonatal units remains limited.

This study aimed to evaluate the usability, user experience, and perceived clinical utility of a paper-based neonatal risk predictor tool derived from an ML model and implemented across 3 Kenyan health facilities.

A postimplementation, cross-sectional usability evaluation was conducted following a 4-month implementation period from August through November 2025. The study was embedded within a longitudinal mixed methods project. Frontline neonatal health care workers (n=10) completed standardized usability instruments, including adapted global usability items (System Usability Scale [SUS]), selected Questionnaire for User Interaction Satisfaction (QUIS) domains, and the Post-Study System Usability Questionnaire (PSSUQ), alongside a project-specific Post-Study Neonatal Utility Questionnaire (PSNUQ). Descriptive statistics (medians, IQRs, and category percentages) were computed. A total of 3 purposively selected neonatal unit leaders participated in semistructured key informant interviews (KIIs), which were analyzed using thematic analysis. Quantitative and qualitative findings were triangulated to contextualize perceptions of usability.

Among participating frontline health care workers, overall perceptions of usability were generally favorable. Around 75% (6/8) of respondents reported being willing to use the tool frequently, and 55% (5/9) indicated confidence in using it independently. Around half (5/10) disagreed that the tool was complex, while 22% (2/9) agreed, indicating moderate polarization in perceived complexity. Median PSSUQ composite scores were below 3 across subscales, reflecting positive usability ratings. Overall, 8 of 10 (80%) respondents agreed that the tool supports early identification of high-risk neonates and improves care prioritization within the first 48 hours. However, workflow integration was workload-sensitive: 40% (4/10) reported an increased documentation burden during periods of high patient volume. KIIs identified staffing shortages, parallel documentation systems, and the importance of administrative endorsement as key structural influences on adoption.

The neonatal risk predictor tool demonstrated acceptable usability, learnability, and perceived clinical relevance across 3 diverse Kenyan facilities. However, variability in perceived complexity and workload sensitivity highlights the importance of structured onboarding, workflow-aligned integration, and context-aware implementation planning. These findings underscore that translating ML-derived predictor models into clinical practice requires not only technical validity but also strong usability and system-level readiness within routine neonatal care settings.

RR2-10.2196/81996.