A Decentralized Partially Observable Decision Model for Recognizing the Multiagent Goal in Simulation Systems

Abstract

Multiagent goal recognition is important in many simulation systems. Many of the existing modeling methods need detailed domain knowledge of agents’ cooperative behaviors and a training dataset to estimate policies. To solve these problems, we propose a novel decentralized partially observable decision model (Dec-POMDM), which models cooperative behaviors by joint policies. In this compact way, we only focus on the distribution of joint policies. Additionally, a model-free algorithm, cooperative colearning based on Sarsa, is exploited to estimate agents’ policies under the assumption of rationality, which makes the training dataset unnecessary. In the inference, considering that the Dec-POMDM is discrete and its state space is large, we implement a marginal filter (MF) under the framework of the Dec-POMDM, where the initial world states and results of actions are uncertain. In the experiments, a new scenario is designed based on the standard predator-prey problem: we increase the number of preys, and our aim is to recognize the real target of predators. Experiment results show that (a) our method recognizes goals well even when they change dynamically; (b) the Dec-POMDM outperforms supervised trained HMMs in terms of precision, recall, and F-measure; and (c) the MF infers goals more efficiently than the particle filter under the framework of the Dec-POMDM.