Evaluation of a computerized aid for creating human behavioral representations of human-computer interaction

Area of Science:

• Human-Computer Interaction
• Cognitive Modeling
• Cognitive Psychology

Background:

• Cognitive models are crucial for understanding user interaction with complex systems.
• Evaluating the reliability of tools used for cognitive modeling is essential for accurate analysis.
• The Cognitive Analysis Tool for Human-Computer Interaction (CAT-HCI) is a computerized tool for creating these models.

Purpose of the Study:

• To empirically evaluate the accuracy and reliability of cognitive models generated using the CAT-HCI tool.
• To compare the consistency and accuracy of models created by domain experts with those created by a double expert.

Main Methods:

• Domain experts modeled their knowledge of specific tasks using the CAT-HCI tool.
• Participants were experts in interacting with a new tactical display for the U.S. Army's Bradley Fighting Vehicle.
• Task models generated by single experts were compared against models developed by a double expert.

Main Results:

• The study found a high degree of consistency among cognitive models created by different single experts.
• The accuracy of the task models generated using CAT-HCI was also found to be high.
• The results suggest that CAT-HCI reliably captures expert knowledge.

Conclusions:

• The CAT-HCI tool is a reliable and accurate method for developing cognitive models of expert task knowledge.
• These findings support the use of CAT-HCI for assessing human-computer interaction complexity and interface productivity.
• The tool's outputs can aid in analyzing interface designs for potential automation of methods.