A Chaotic Neuron and its Ability to Prevent Overfitting

Abstract

Chaotic neuron is a neural model based on chaos theory, which combines the complex dynamic behavior of biological neurons with the characteristics of chaotic systems. Inspired by the chaotic firing characteristics of biological neurons, a novel chaotic neuron model and its response activation function LMCU are proposed in this paper. Based on one-dimensional chaotic mapping, this chaotic neuron model takes the emissivity of chaotic firing characteristics of biological neurons as its response output, so that it has the nonlinear response and chaotic characteristics of biological neurons. Different from the traditional neuron model, it makes full use of the nonlinear dynamics of the chaotic system to achieve the activation output. In this paper, we apply the proposed chaotic neurons to artificial neural networks by using LeNet-5 models on MNIST and CIFAR-10 datasets, and compare them with common activation functions. The application of chaotic neurons can effectively reduce the overfitting phenomenon of artificial neural network, significantly reduce the generalization error of the model, and greatly improve the overall performance of artificial neural network. The innovative design of this chaotic neuron model provides a new cornerstone for the future development of artificial neural networks.