Evaluating probe techniques and a situated theory of situation awareness

Probe techniques for measuring situation awareness (SA) vary in whether scenarios are paused and displays visible while questions are presented. We examined which technique is least intrusive on workload and performance in air traffic control, and which is most sensitive at capturing differences in SA when automation varies. We also tested predictions from the situated SA theory, which holds that operators offload specific and low-priority information onto displays to limit internal processing. To accomplish these goals, Experiments 1 and 2 manipulated whether radar displays were visible and scenarios paused during queries. Experiment 2 also manipulated the amount of automation by varying the percentage of aircraft equipped with NextGen tools. We found all probe techniques were equally sensitive at capturing SA differences for different levels of equipage, but those that paused scenarios were least intrusive. Moreover, consistent with situated SA, blanking displays impaired ability to answer questions about specific but not general information. Experiment 3 recorded eye gaze frequency and duration during queries when scenarios were visible and not paused and, as predicted by situated SA, found participants were more likely to look at radar displays while answering specific and low-priority questions than general and high-priority questions. (PsycINFO Database Record