Collaborate and explain on-the-fly: knowledge-based reasoning and learning in ad hoc teamwork

This paper focuses on ad hoc teamwork, the problem of enabling an AI agent to collaborate with other agents without prior coordination. Methods considered state of the art for ad hoc teamwork formulate it primarily as a learning problem, using a large labeled dataset of different situations to model the action choices of other agents (or agent types) and determine the actions of the ad hoc agent. Such datasets are not readily available in practical domains, and these methods lack transparency and make it difficult to rapidly revise existing knowledge (or models) in response to changes in the domain, team composition, or agents' capabilities. Our architecture for ad hoc teamwork embeds the principles of refinement, ecological rationality, interactive learning, and explainable agency, leveraging the complementary strengths of knowledge-based and data-driven methods for reasoning and learning. Specifically, for any given goal, our architecture enables an ad hoc AI agent to determine its actions through non-monotonic logical reasoning with: (a) prior domain-specific commonsense knowledge; (b) models learned and revised rapidly to predict the behavior of other agents; and (c) anticipated abstract future goals based on generic knowledge of similar situations in a pretrained Large Language Model. In addition, the ad hoc agent processes natural language descriptions and observations of other agents' behavior, using a combination of a pretrained Large Language Model and decision-tree induction to incrementally acquire and revise knowledge in the form of objects, actions, and axioms that govern domain dynamics. Furthermore, the ad hoc agent generates relational descriptions as on-demand explanations of its decisions and beliefs, and those of other agents, in response to various types of questions. We ground and experimentally evaluate the capabilities of our architecture in VirtualHome, a realistic, physics-based 3D simulation environment. We demonstrate reliable, efficient, transparent, and scalable performance, providing a substantial improvement in performance compared with a purely knowledge-based baseline, and comparable or better performance than a purely data-driven baseline while using orders of magnitude fewer resources.