Construction in a Simulated Environment Using Temporal Goal Sequencing and Reinforcement Learning

Abstract

A behavior-based architecture ( ConAg) with a connectionist action selection mechanism is introduced that enables a society of autonomous agents to construct arbitrary structures in their simulated two-dimensional world. Construction in this environment involves the agents picking up colored discs and dropping them at incomplete parts of the structure being built. The ConAg architecture provides both reactive behaviors which are used to maintain the viability of the agent and navigational planning behaviors that are used for construction. The action selection mechanism enables learning the sequence of behaviors required for construction by reinforcement learning. The navigational planning behaviors use a grid-based representation of the world. The shape of the structure to be built is also encoded on an internal spatial map. Path planning is implemented by spreading activations on sets of grid-based maps so that the agents perform the construction task efficiently. Construction of arbitrary structures is supported by temporal sequencing of goals. We present simulation results that demonstrate the performance of the architecture and algorithms.