Digital innovations and health information systems: Lessons learned from implementing ALMANACH in Nigeria

In low- and middle-income settings, routine health information systems (RHIS) and digital health projects often coexist, but their epidemiological outputs are rarely compared-even though integrating digital tools could strengthen RHIS by reducing reporting challenges. We conducted a retrospective, facility-level analysis of quarterly data (2017-2021) from Adamawa State, Nigeria, comparing ALMANACH, a clinical decision support system used in primary health care, with the state RHIS for malaria, pneumonia, gastrointestinal disorders, and measles in children under five years of age. The primary outcome was the facility-aggregated quarterly absolute relative difference (ARD) between the two reporting systems; temporal trends and facility-level heterogeneity were also assessed. Paired non-parametric tests, effect sizes with 95% confidence intervals (95% CI), and linear mixed-effects models accounted for clustering and repeated measurements. Across the study period, ALMANACH reported fewer cases than RHIS for malaria (116,018 vs 233,548; ARD 80.6%, 95% CI: 64.4-89.6, p < 0.01), gastrointestinal disorders (43,003 vs 62,412; ARD 32.6%, 95% CI: 17.1-47.4, p < 0.01), and pneumonia (20,980 vs 29,416; ARD 34.0%, 95% CI: 32.3-48.0, p < 0.05), but more measles cases (4,355 vs 2,508; ARD 86.2%, 95% CI: 45.6-162.8, p < 0.01). Mixed-effects models showed that RHIS recorded, per facility-quarter, on average 40.2 (95% CI: 31.1-49.3) more malaria, 6.6 (95% CI: 4.3-8.0) more gastrointestinal disorders, and 2.9 (95% CI: 1.2-4.6) more pneumonia cases than ALMANACH (all p < 0.01), while ALMANACH reported 0.6 (95% CI: 0.07-1.2) more measles cases (p < 0.05). Divergence widened as ALMANACH scaled up. Without a gold standard, these results quantify discrepancies without implying superiority and highlight the need for data integration, harmonized case definitions, and stronger data-use practices. They underscore the importance of sustained stakeholder engagement and user involvement for effective digital health scale-up in resource-limited settings.