Usability testing of the Philippine Electronic National Newborn Hearing Screening Registry (ENNHSR)

The study determined the usability of the online and offline versions of the Philippine Electronic National Newborn Hearing Screening Registry (ENNHSR) as well as user perspectives and satisfaction on the training modules and the online and offline systems. The steps in creating the systems, training modules, and evaluation of the user training manual and video training modules, accuracy and time and motion studies on data entry as well as determination of user perspectives and satisfaction were the specific objectives.

With the combined efforts of the staff of Newborn Hearing Screening Reference Center (NHSRC), Philippine National Ear Institute (PNEI) and the National Telehealth Center (NTHC) of the National Institutes of Health UP Manila, the development of the online and offline versions of the ENNHSR took six (6) months from January 2021 to June 2021 to complete. Creation of the user manual and training modules took three (3) months from July 2021 to September 2021. The pilot of the systems was carried out in 2 Zoom Conferencing sessions with the participation of 28 existing certified newborn hearing center users with different roles, backgrounds, demographics from all over the Philippines. Written evaluation as well as focused group discussions on the training modules and the database were conducted during the sessions. Effectivity of the training modules was determined using a 10-point learning check. The time and accuracies in encoding each data field per user were also determined.

All 28 participants were able to attend and actively participate in the required Zoom Conferencing sessions as well as submit the 2 evaluation surveys for the training modules and the ENNHSR. During the learning check 93% or 26 out of 28 passed. The surgical intervention module took the longest time to encode while the fastest module to complete was for speech therapy. The average mean time to complete all modules was 3382 s or around 57 min while the time range was between 32 and 104 min. A screener would need 18 min while an implant programmer who is a clinical audiologist would need 52 min to enter data. The accuracy in encoding patient data was 92% while hearing screening results was 88.64%. The system usability scale (SUS) score of ENNHSR was computed at 75.5 which was the average of individual SUS scores, falling within grade B or 74.1 to 77.1 as its corresponding numerical score range in percentile. Most of the participants noted that it was easy to find patient data, results and that it was streamlined with easy to track information.

Data gathering and analysis both play important roles in health management, policy implementation and quality assurance. We were able to uncover areas where the system performed well - effectively, efficiently, and with satisfaction. We realize that all the possible problems cannot be detected with a small number of participants and variety in information. This testing will serve as both a means to record or benchmark current usability, but also to identify areas where improvements must be made.