The significance of a usability evaluation of an emerging laboratory order entry system

To assess the usability of an emerging POE system, OM/Lab, for the computer-supported ordering of laboratory tests. We were more specifically interested in the relation of the usability problems detected in the user testing sessions with the order behaviour in terms of efficiency and errors in ordering.

A cognitive walkthrough of the OM/Lab system was conducted by two analysts using four real-life scenarios for ordering laboratory tests, which were reviewed for comprehensiveness by an expert clinician. Thereafter, the OM/Lab system was evaluated on its usability in testing sessions with seven potential end-users of the system performing these same four scenarios. The results of these end-user testing sessions were used to analyze the effect of usability flaws on the quality of ordering in terms of omissions, errors in orders and cancelled orders.

The analyses revealed a total of 33 usability problems, of which 25 problems were revealed both by the cognitive walkthrough and in the end-user sessions. These 25 usability problems indeed led to inefficient order behaviour, omissions and errors in orders and even to cancelled orders.

Our results revealed that the OM/Lab system suffered from a high number of usability flaws. The interface design flaws were, among other things, related to misallocation of buttons on the screen, incomprehensibility of button labels and feedback containing no relevant information to the user about the cause of errors made and consequences of a user's action. Additionally, our user test session results indicated that the OM/Lab system also suffered from user interaction problems of a more socio-technical nature. These sessions revealed, among other things, that the more specific action sequences to be executed within the ordering model of the OM/Lab system did not correspond to the daily working routines of end-users and that the grouping of laboratory tests within clusters did not match the paper-based order forms. The seemingly negative effects of these usability flaws on the quality of ordering and the inefficiency of work processes during the pilot implementation finally lead to withdrawal of the OM/Lab system from clinical practice. Though the system implementation failed, our usability study brought research on (re)designing and evaluating clinical computer applications at a higher status in our institution. It is now recognized that usability evaluation studies that will support good quality of clinical practice are highly important.