A pilot study on usability analysis of emergency department information system by nurses

Employing new health information technologies while concurrently providing quality patient care and reducing risk is a major challenge in all health care sectors. In this study, we investigated the usability gaps in the Emergency Department Information System (EDIS) as ten nurses differentiated by two experience levels, namely six expert nurses and four novice nurses, completed two lists of nine scenario-based tasks.

Standard usability tests using video analysis, including four sets of performance measures, a task completion survey, the system usability scale (SUS), and sub-task analysis were conducted in order to analyze usability gaps between the two nurse groups.

A varying degree of usability gaps were observed between the expert and novice nurse groups, as novice nurses completed the tasks both less efficiently, and expressed less satisfaction with the EDIS. The most interesting finding in this study was the result of 'percent task success rate,' the clearest performance measure, with no substantial difference observed between the two nurse groups. Geometric mean values between expert and novice nurse groups for this measure were 60% vs. 62% in scenario 1 and 66% vs. 55% in scenario 2 respectively, while there were some marginal to substantial gaps observed in other performance measures. In addition to performance measures and the SUS, sub-task analysis highlighted navigation pattern differences between users, regardless of experience level.

This study will serve as a baseline study for a future comparative usability evaluation of EDIS in other institutions with similar clinical settings.