Learning Attentional and Gated Communication via Curiosity

Due to the partial observability in decentralized multi-agent systems, communication is critical for cooperation. Furthermore, the ability to decide when and whom to communicate is important to achieve efficient communication. However, the existing methods are typically driven by extrinsic rewards. Hence, when the reward from environment is sparse, delayed, or noisy, the communication performance of these methods would be restricted. Furthermore, it would introduce additional difficulty named credit assignment when using extrinsic reward to train communication and sample policies together. To tackle these difficulties, we introduce the mechanism of intrinsic motivation from psychology to multi-agent communication. We hold the view that the observations with more uncertainty and curiosity are more valuable for communication. It can help agent find useful information from observations. It is a good complement to existing extrinsic driven methods. Concretely, at sending end, we learn a curiosity from local observations to model the communication importance. Then, we design a heuristic mechanism to prune unnecessary messages. It can solve the problem of when to communicate. Then, the ability to gate unnecessary message can reduce the cost and improve the efficiency of communication, which is important to apply to real-world scenarios. Furthermore, at receiving end, we utilize the intrinsic importance to differentiate information, which can be helpful for local decisions. It could solve the problem of whom to communicate. The ability to pay attention to useful information can efficiently improve the performance of communication behaviors. At last, we evaluate our method on a variety of multi-agent scenarios. The experiments of full communication demonstrate that the curiosity is capable to model the communication importance, and the results of gated communication further prove the conclusion.