DESTformer: A Transformer Based on Explicit Seasonal–Trend Decomposition for Long-Term Series Forecasting
-
Published:2023-09-20
Issue:18
Volume:13
Page:10505
-
ISSN:2076-3417
-
Container-title:Applied Sciences
-
language:en
-
Short-container-title:Applied Sciences
Author:
Wang Yajun1ORCID, Zhu Jianping2ORCID, Kang Renke1
Affiliation:
1. School of Mechanical Engineering, Dalian University of Technology, Dalian 116024, China 2. School of Computer Science and Technology, Dalian University of Technology, Dalian 116024, China
Abstract
Seasonal–trend-decomposed transformer has empowered long-term time series forecasting via capturing global temporal dependencies (e.g., period-based dependencies) in disentangled temporal patterns. However, existing methods design various auto-correlation or attention mechanisms in the seasonal view while ignoring the fine-grained temporal patterns in the trend view in the series decomposition component, which causes an information utilization bottleneck. To this end, a Transformer-based seasonal–trend decomposition methodology with a multi-scale attention mechanism in the trend view and a multi-view attention mechanism in the seasonal view is proposed, called DESTformer. Specifically, rather than utilizing the moving average operation in obtaining trend data, a frequency domain transform is first applied to extract seasonal (high-frequency) and trend (low-frequency) components, explicitly capturing different temporal patterns in both seasonal and trend views. For the trend component, a multi-scale attention mechanism is designed to capture fine-grained sub-trends under different receptive fields. For the seasonal component, instead of the frequency-only attention mechanism, a multi-view frequency domain (i.e., frequency, amplitude, and phase) attention mechanism is designed to enhance the ability to capture the complex periodic changes. Extensive experiments are conducted on six benchmark datasets covering five practical applications: energy, transportation, economics, weather, and disease. Compared to the state-of-the-art FEDformer, our model shows reduced MSE and MAE by averages of 6.5% and 3.7%, respectively. Such experimental results verify the effectiveness of our method and point out a new way towards handling trends and seasonal patterns in long-term time series forecasting tasks.
Funder
National Defense Basic Scientific Research Program of China
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference39 articles.
1. Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., and Zhang, W. (2021, January 2–9). Informer: Beyond efficient transformer for long sequence time-series forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada. 2. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting;Wu;Adv. Neural Inf. Process. Syst.,2021 3. Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., and Jin, R. (2022, January 17–23). Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. Proceedings of the International Conference on Machine Learning, PMLR, Baltimore, MD, USA. 4. Guo, C., Li, D., and Chen, X. (2023). Unequal Interval Dynamic Traffic Flow Prediction with Singular Point Detection. Appl. Sci., 13. 5. Han, L., Du, B., Sun, L., Fu, Y., Lv, Y., and Xiong, H. (April, January 13). Dynamic and multi-faceted spatio-temporal deep learning for traffic speed forecasting. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Virtual.
|
|