Multiple-Reservoir Hierarchical Echo State Network

Abstract

Leaky Integrator Echo State Network (Leaky-ESN) is a useful training method for handling time series prediction problems. However, the singular coupling of all neurons in the reservoir makes Leaky-ESN less effective for sophisticated learning tasks. In this paper, we propose a new improvement to the Leaky-ESN model called the Multiple-Reservoir Hierarchical Echo State Network (MH-ESN). By introducing a new mechanism for constructing the reservoir, the efficiency of the network in handling training tasks is improved. The hierarchical structure is used in the process of constructing the reservoir mechanism of MH-ESN. The MH-ESN consists of multiple layers, each comprising a multi-reservoir echo state network model. The sub-reservoirs within each layer are linked via principal neurons, which mimics the functioning of a biological neural network. As a result, the coupling among neurons in the reservoir is decreased, and the internal dynamics of the reservoir are improved. Based on the analysis results, the MH-ESN exhibits significantly better prediction accuracy than Leaky-ESN for complex time series prediction.