Intelligent Data Analysis · 2023 · 13 citations · 6 references
ComputingComputer ScienceForecastingMachine LearningTime Series AnalysisComputational Time Series AnalysisDeep LearningTime Series ComputationTime Series ForecastingTime Series ModelingTime Series PredictionDeep Learning For ForecastingDeep Learning For Time Series ForecastingDeep Learning-based ForecastingForecasting With Sparse DataNeural ForecastingTraffic PredictionTransformer ModelsTransformer-based Prediction Models
With the continuous development of deep learning, long sequence time-series forecasting (LSTF) has attracted more and more attention in power consumption prediction, traffic prediction and stock prediction. In recent studies, various improved models of Transformer are favored. While these models have made breakthroughs in reducing the time and space complexity of Transformer, there are still some problems, such as the predictive power of the improved model being slightly lower than that of Transformer. And these models ignore the importance of special values in the time series. To solve these problems, we designed a more concise network named Resformer, which has four significant characteristics: (1) The fully sparse self-attention mechanism achieves O(𝐿𝑙𝑜𝑔𝐿) time complexity. (2) The AMS module is used to process the special values of time series and has comparable performance on sequences dependency alignment. (3) Using quadratic linear transformation, a simple LT module is designed to replace the self-attention mechanism. It effectively reduces redundant information. (4) The DistPooling method based on data distribution is proposed to suppress redundant information and noise. A large number of experiments on real data sets show that the Resformer method is superior to the existing improved model and standard Transformer method.
6
Sean J. Taylor, Benjamin Letham · The American Statistician · 2017 · 2.3K citations
Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks
Guokun Lai, Wei-Cheng Chang, Yiming Yang et al. · 2018 · 2.1K citations
Fisher Yu, Vladlen Koltun, Thomas Funkhouser · 2017 · 1.8K citations
Stock Price Prediction Using the ARIMA Model
Aderemi O. Adewumi, C. K. Ayo · 2014 · 1K citations
Business, Accounting, Finance +15