Reinforcement Learning in Deep Spiking Neural Networks with Eligibility Traces and Modifying the Threshold Parameter

Abstract

Abstract Desirable features of Spiking Neural Networks (SNNs) such as biological plausibility, event-based information processing, and energy efficiency have led to their widespread application in various machine learning domains in recent years. The reinforcement learning, inspired by the human visual system, has also gained popularity recently. This paper proposed reinforcement learning in a deep SNN which presents the idea of using eligibility traces to apply reward signals. We also focus on a locally connected SNN that uses a reinforcement learning based on Spike-Timing-Dependent Plasticity (STDP) called R-STDP for pattern learning. Locally connected networks, exhibit a closer resemblance to the biological visual system by extracting key topological features of the image. By introducing this idea and modifying several intrinsic parameters of spiking neurons (adjusting the voltage threshold, as well as the membrane time constant), the model achieves an 87.84% accuracy on MNIST dataset, showing significant improvement compared to the previous similar model.