Cognitive computer-based video analysis: its application in assessing the usability of medical systems

This paper describes a methodology, based on cognitive research, for assessing the usability of medical computing systems. The issue of developing appropriate evaluation tools, both for use in the design process and for analysis of end products, is beginning to be recognized as being of great importance. In this paper, the use of video recording for collecting empirical data on system usability is detailed. The techniques described allow for the collection of an integrated data set consisting of the transcripts of physicians as they "think aloud" in interacting with a medical system, along with video records of user-computer interaction. The use of coding methodologies and a computer-based annotation system for the analysis of video data are described. Our preliminary experience indicates that this methodology offers a powerful way for assessing physicians' informational needs. Implications for the development and evaluation of medical information systems are discussed.