Temporal Chain Network With Intuitive Attention Mechanism for Long-Term Series Forecasting

Abstract

Long-term series forecasting (LTSF) plays an important role for real-world application in the economic, the weather and the industrial process. At present, many Transformer-based methods have made promising progress. However, the nature of the permutation-invariant self-attention mechanism inevitably results in temporal information loss, which hinder the prediction performance of the Transformer-based LTSF methods. Therefore, this paper proposed a novel temporal chain network with intuitive attention mechanism (TCNet) for long-term series forecasting. Based on the chain forward propagation structure of time series, a one-way chain graph neural network (GNN) is constructed to avoid the permutation-invariance of the self-attention mechanism. Meanwhile, based on the natural forgetting mechanism of time series, the prior intuitive attention is proposed as the edge weight (attention) for information propagation and then the series model is obtained by the GNN. Furthermore, the proposed method achieves the LTSF task by linear forecasting of the trend component and nonlinear forecasting of the seasonal component of the series. Extensive experiments on five benchmarks and a real-world chemical process dataset are conducted to demonstrate the effectiveness of the TCNet. Comparison experiment results show that the TCNet achieves the state-of-the-art results compared to current baselines and reduces average prediction error by 9.52% and 5.97% on Transformer-based and non-Transformer-based multivariate LTSF baselines, respectively. Moreover, the temporal information loss for the TCNet due to the permutation-invariance of the self-attention mechanism is not the main reason hindering Transformer-based LTSF methods, whose bottleneck comes mainly from the complex architecture of the decoder.