Applying reinforcement learning to an insurgency Agent-based Simulation

Abstract

A requirement of an Agent-based Simulation (ABS) is that the agents must be able to adapt to their environment. Many ABSs achieve this adaption through simple threshold equations due to the complexity of incorporating more sophisticated approaches. Threshold equations are when an agent behavior changes because a numeric property of the agent goes above or below a certain threshold value. Threshold equations do not guarantee that the agents will learn what is best for them. Reinforcement learning is an artificial intelligence approach that has been extensively applied to multi-agent systems but there is very little in the literature on its application to ABS. Reinforcement learning has previously been applied to discrete-event simulations with promising results; thus, reinforcement learning is a good candidate for use within an Agent-based Modeling and Simulation (ABMS) environment. This paper uses an established insurgency case study to show some of the consequences of applying reinforcement learning to ABMS, for example, determining whether any actual learning has occurred. The case study was developed using the Repast Simphony software package.