Rasmussen and the boundaries of empirical evaluation

In this special issue, many of the papers focus on Rasmussen's analytic contributions to the understanding of work in complex sociotechnical systems. Work is analysed for the purpose of developing new designs that can improve the nature of that work. The evaluation of such designs was a key part of Rasmussen's program, yet he was often sceptical of the claims made for the generalizability of empirical studies. To tackle this problem, he extended his work analysis framework to provide a way of thinking about empirical evaluation. As authors of this paper, we come from two different backgrounds-systems engineering in the case of Burns, and engineering psychology in the case of Sanderson-and over the decades of our respective research programs, we have both performed many empirical investigations: field investigations, simulation studies, and behavioural laboratory experiments. Rasmussen's scepticism-and his writings on the issue-have stimulated and shaped our own research. In this brief paper we present our interpretation of Rasmussen's perspective, we provide examples how our research sits within Rasmussen's framework of constraints defining boundary conditions for experiments, and we draw conclusions for the future.