Usability evaluation of a laboratory order entry system: cognitive walkthrough and think aloud combined

Though designing physician order entry (POE) systems has been a highly discussed research topic in the last decade, it seems that engineering POE systems that truly optimise the quality and efficiency of ordering is still a challenge. This study addresses a usability evaluation of an emerging POE for the electronically ordering of laboratory tests. By applying two complementary cognitive approaches, specifically the cognitive walkthrough and think aloud method with seven potential end users of the system, we analyzed usability problems in the prototype system and their potential effect on the quality of orders in terms of efficiency and errors in ordering. The cognitive walkthrough provided a coding scheme that was used to analyse in more detail usability errors encountered during the think aloud tests with the seven end users. The analyses revealed a total of 33 usability problems, which indeed led to inefficiency, omissions in ordering and even to cancelled orders. Most of these usability problems referred to incomprehensiveness of required actions by the user and incomprehensiveness of text used in the system. Next to the discussion of the reasons for these usability problems, the surplus value of the think aloud method as supplementary to the cognitive walkthrough in evaluating physician order entry systems is outlined in this paper.