Cognitive evaluation of the user interface and vocabulary of an outpatient information system

This paper describes an innovative approach to the evaluation of the user interface and vocabulary of a medical information system. The use of video recording for collecting usability data is detailed. The technique employed involves the collection of data consisting of transcripts of physicians as they "think aloud" while interacting with the system, along with a video record of the complete user-computer interaction. Using methods of analysis from cognitive science, the study was able to distinguish the source of physician problems in using the system's interface and in interacting with its controlled medical vocabulary. Analysis of the protocols indicated that all subjects encountered several generic problems, the most common ones indicative of a need for greater consistency in the interface design. Based on this evaluation, parts of the user interface have been re-implemented in an ongoing process of iterative system development.