Sparse self-attention guided generative adversarial networks for time-series generation

Abstract

AbstractRemarkable progress has been achieved in generative modeling for time-series data, where the dominating models are generally generative adversarial networks (GANs) based on deep recurrent or convolutional neural networks. Most existing GANs for time-series generation focus on preserving correlations across time. Although these models may help in capturing long-term dependencies, their capacity to pay varying degrees of attention over different time steps is inadequate. In this paper, we propose SparseGAN, a novel sparse self-attention-based GANs that allows for attention-driven, long-memory modeling for regular and irregular time-series generation through learned embedding space. This way, it can yield a more informative representation for time-series generation while using original data for supervision. We evaluate the effectiveness of the proposed model using synthetic and real-world datasets. The experimental findings indicate that forecasting models trained on SparseGAN-generated data perform comparably to forecasting models trained on real data for both regularly and irregularly sampled time series. Moreover, the results demonstrate that our proposed generative model is superior to the current state-of-the-art models for data augmentation in the low-resource regime and introduces a novel method for generating realistic synthetic time-series data by leveraging long-term structural and temporal information.