T-ARX Model for Nonlinear Discrete System Prediction Based on A Hybrid Modelling Method

Abstract

Abstract A suited model is meaningful for controlling a plant. A nonlinear model composed of deep neural network impedes the application of the mature controlling methods because of its complex formation, though it may predict precisely. To model a nonlinear and dynamic system with local-linearity and global-nonlinearity, a hybrid model named T-ARX (Transformer Auto-Regression model with eXogenous variables) with good long-term prediction accuracy is proposed in this paper. The T-ARX model is one kind of SD-ARX (State Dependent ARX) model with the coefficients of regression inputs estimated by a deep Transformer network, so that the nonlinear dynamics of plant could be captured. Meanwhile, the model takes state and action sampled from a period of time as its inputs and possesses the pseudo linear structure. When its coefficients are fixed at the sample time, it can be reformed as a discrete state-space model which is quite meaningful for controller design. Because the mask is applied to the input sequence, the model can be trained through parallel schemes, thus the short-term and long-term prediction ability can be trained. To demonstrate the feasibility of the proposed model in both fast-responding and slow-responding system, four experiments were conducted on a quad-rotor helicopter, a maglev ball system, a simulated inverted pendulum, and the Box-Jenkins gas furnace data, respectively. The results demonstrate the superiority of the proposed model over RBF-ARX model and some others in one-step-ahead prediction and multi-step-ahead prediction and show the feasibility of this model.