Online adaptive learning for team strategies in multi-agent systems

Abstract

During mission execution in military applications, the TRADOC Pamphlet 525-66 Battle Command and Battle Space Awareness capabilities prescribe expectations that networked teams will perform in a reliable manner under changing mission requirements and changing team and individual objectives. In this paper we first present an overall view for dynamical decision-making in teams, both cooperative and competitive. Strategies for team decision problems, including optimal control, N-player games ( H∞ control, non-zero sum) and so on are normally solved offline by solving associated matrix equations such as the coupled Riccati equations or coupled Hamilton–Jacobi equations. However, using that approach, players cannot change their objectives online in real time without calling for a completely new offline solution for the new strategies. Therefore, in this paper we give a method for learning optimal team strategies online in real time as team dynamical play unfolds. In the linear quadratic regulator case, for instance, the method learns the coupled Riccati equations solution online without ever solving the coupled Riccati equations. This allows for truly dynamical team decisions where objective functions can change in real time and the system dynamics can be time-varying.