A Systematic Review and Taxonomy of Human-Agent Teaming Testbeds

ObjectiveWe developed a taxonomy for human-agent teams (HATs) and conducted a literature review of existing HAT testbeds using our proposed taxonomy.BackgroundWith the increasing interest in HATs, numerous research studies in this field have utilized different testbeds. Despite this, there is a lack of comprehensive understanding regarding the capabilities and limitations of the existing testbeds.MethodWe first developed a taxonomy for HATs by modifying the existing framework for classifying human teams. Our proposed taxonomy comprises ten attributes. Subsequently, using the taxonomy, we analyzed 103 testbeds identified from 235 empirical research studies. After coding each testbed, we conducted frequency analyses on each attribute to determine the distribution of the testbeds.ResultsRegarding team composition, the majority of testbeds afford single human participants paired with few agents, typically in subordinate roles. Also, in most testbeds, the leadership structure is designated, with humans assuming leadership roles, or none. The communication dynamics present an area for further exploration, especially with larger team sizes. Additionally, nearly all reviewed testbeds focus on long-term teams, overlooking dynamics in ad hoc teams, which are common in real-world settings.ConclusionOur findings underscore the importance of further research into diverse team attributes, such as team composition, leadership structure, communication structure, direction, and medium. It would facilitate a deeper understanding of complex team dynamics in HATs and lead to designing effective teams.ApplicationThe current study would be valuable for discussing future research directions when developing new testbeds or designing novel experiments leveraging existing ones.