Design and evaluation of multimedia stimuli to evoke clinical concepts

Continuous speech recognition systems have the potential to facilitate clinical data entry, but evaluating them rigorously is difficult. We describe a tool to aid evaluators of such systems. The tool is a HyperCard stack with stimuli consisting of pictures, sounds and the minimum of words to evoke 20 QMR physical findings. Despite using up to four different stimuli to communicate each finding and piloting the material on six subjects, eight test subjects made a total of 66 errors (42%) in interpreting the 20 sets of stimuli, of which 22 errors (14%) were serious. These results are relevant to those designing interfaces for decision-support, tutorial and student testing systems.