Multitasking in RNN: an analysis exploring the combination of simple tasks

Abstract

Abstract The brain and artificial neural networks are capable of performing multiple tasks. The mechanisms through which simultaneous tasks are performed by the same set of units in the brain are not yet entirely clear. Such systems can be modular or mixed selective through some variables such as sensory stimulus. Recurrent neural networks can help to a better understanding of those mechanisms. Based on simple tasks studied previously in Jarne 2020 arXiv Preprint 2005.13074, multitasking networks were trained and analyzed. In present work, a simple model that can perform multiple tasks using a contextual signal was studied, trying to illuminate mechanisms similar to those that could occur in biological brains. Backpropagation through time allows training networks with multitasking, but the realizations obtained are not unique. Different realizations for the same set of tasks are possible. Here the analysis of the dynamics and emergent behavior of their units is presented. The goal is to try to describe better the models used to describe different processes in the cortex.