Reinforcement learning in biological systems for adaptive regulation

Abstract

Abstract The adaptive control of complex biological systems remains unclear despite extensive research on their regulatory networks. We recently reported that epigenetic regulation of gene expression may be a learning process, in which amplification-and-decay cycles optimize expression patterns while basically maintaining current patterns. Here, we show that various biological processes, such as intestinal immunity, population dynamics, chemotaxis, and self-organization, are also characterized as reinforcement learning (RL) processes. An appropriate population balance is established autonomously through symmetric competitive amplification and decay, which is a biologically plausible RL process. Monte Carlo simulations of predator-prey numbers show that population dynamics based on this RL process enable the sustainability of predators and reproduce fluctuations with a phase delay when humans hunt prey more preferentially than predators. Another example is a random walk controlling step-length (s-rw), which allows the agent to approach the target position with a Levy walk trajectory. In addition, shortcut paths in a maze are autonomously generated by s-rw using a moving-direction policy or bias, which is optimized through another RL on a longer timescale. Furthermore, by applying s-rw to reaction-diffusion theory, Turing patterns can be self-organized. The RL process, expressed by a common mathematical equation, enables the adaptability of biological systems.