Control of chaos in Frenkel–Kontorova model using reinforcement learning*

Abstract

It is shown that we can control spatiotemporal chaos in the Frenkel–Kontorova (FK) model by a model-free control method based on reinforcement learning. The method uses Q-learning to find optimal control strategies based on the reward feedback from the environment that maximizes its performance. The optimal control strategies are recorded in a Q-table and then employed to implement controllers. The advantage of the method is that it does not require an explicit knowledge of the system, target states, and unstable periodic orbits. All that we need is the parameters that we are trying to control and an unknown simulation model that represents the interactive environment. To control the FK model, we employ the perturbation policy on two different kinds of parameters, i.e., the pendulum lengths and the phase angles. We show that both of the two perturbation techniques, i.e., changing the lengths and changing their phase angles, can suppress chaos in the system and make it create the periodic patterns. The form of patterns depends on the initial values of the angular displacements and velocities. In particular, we show that the pinning control strategy, which only changes a small number of lengths or phase angles, can be put into effect.