Paper Infomation
Multivariate Time Series Forecasting Based on Pre-trained Temporal Encoders and Graph Neural Networks
Full Text(PDF, 206KB)
Author: Yu Li, Xiaoxiao Wu
Abstract: Multivariate time series widely exist in many real-world systems such as finance, transportation, and social networks. Forecasting such data requires modeling both the temporal dynamics within individual sequences and the dependency relationships among different entities. In this paper, we propose a multimodal multivariate forecasting framework (MMPF) that jointly models temporal and relational information. In the temporal dimension, temporal neural networks (TNNs) are employed to capture short-term fluctuations and long-term trends of time series. In the spatial dimension, a graph structure is constructed to represent the relationships among entities, and graph neural networks (GNNs) are used to learn the structural dependencies. In addition, external news text information is incorporated to provide complementary semantic signals. The extracted multimodal features are finally fused by a multilayer perceptron (MLP) for prediction. Experimental results demonstrate that multimodal feature fusion can effectively improve the forecasting performance of multivariate time series.
Keywords: Multivariate Time Series; Graph Neural Networks; Multimodal Feature Fusion; Time Series Forecasting; Cryptocurrency
References:
[1] Corbet S, Lucey B, Urquhart A, Yarovaya L. Cryptocurrencies as a financial asset: A systematic analysis[J]. International Review of Financial Analysis, 2019, 62: 182–199.
[2] Wu Z, Pan S, Long G, Jiang J, Chang X, Zhang C. Connecting the dots: multivariate time series forecasting with graph neural networks[J]. arXiv preprint, 2020: 753–763.
[3] Sun L, Liu X, Liu G, Yang Z, Liu S, Li Y, Wu X. Deep learning-based multivariate load forecasting for integrated energy systems[J]. International Journal of Low-Carbon Technologies, 2025, 20: 957–964.
[4] Ng K W, Mubang F, Hall L O, Skvoretz J, Iamnitchi A. Experimental evaluation of baselines for forecasting social media time series[J]. PLoS One, 2023, 12: 1–26.
[5] Antonakakis N, Chatziantoniou I, Gabauer D. Cryptocurrency market contagion: market uncertainty, market complexity, and dynamic portfolios[J]. Journal of International Financial Markets, Institutions and Money, 2019, 61: 37–51.
[6] Bialkowski J, Gottschalk K, Wisniewski T P. High policy uncertainty and low implied market volatility: an academic puzzle?[J]. Journal of Financial Economics, 2022, 143(3): 1185–1208.
[7] Hackethal A, Inderst R, Meyer S. The characteristics and portfolio behavior of Bitcoin investors: evidence from indirect cryptocurrency investments[J]. Review of Finance, 2021, 26(4): 855–898.
[8] Liu J, Serletis A. Volatility and dependence in cryptocurrency and financial markets: a copula approach[J]. Studies in Nonlinear Dynamics Econometrics, 2024, 28(1): 119–149.
[9] Fung K, et al. More to cryptos than Bitcoin: A GARCH modelling of heterogeneous cryptocurrencies[J]. Finance Research Letters, 2022, 47.
[10] Jlassi N B, et al. Subsample analysis of stock market–cryptocurrency returns tail dependence: A copula approach for the tails[J]. Finance Research Letters, 2023, 58.
[11] Zhang J, He Q Z. Dynamic cross-market volatility spillover based on MSV model: Evidence from Bitcoin, gold, crude oil, and stock markets[J]. Complexity, 2021: 1–8.
[12] Seabe P L, Madzivhandila T, Odhiambo N M. Forecasting cryptocurrency prices using LSTM, GRU, and bi-directional LSTM: A deep learning approach[J]. Fractal and Fractional, 2023, 7(2).
[13] Guo H, Li X, Li Z, Chen Y. Bitcoin price forecasting: A perspective of underlying blockchain transactions[J]. Decision Support Systems, 2021, 151.
[14] Das A, Kong W, Sen R, Zhou Y. A decoder-only foundation model for time-series forecasting[C]// Proceedings of the 41st International Conference on Machine Learning (ICML). 2024: 10148–10167.
[15] Ye Z, et al. A stacking ensemble deep learning model for Bitcoin price prediction using Twitter comments on Bitcoin[J]. Mathematics, 2022, 10(8).
[16] Tanwar S, et al. Deep learning-based cryptocurrency price prediction scheme with inter-dependent relations[J]. IEEE Access, 2021, 9: 138633– 138646.
[17] Zhou Y, et al. Forecasting cryptocurrency volatility: A novel framework based on the evolving multiscale graph neural network[J]. Financial Innovation, 2025, 11(1): 1–52.
[18] Zhao L, Chen Q, Huang Z, Xu J. Long-term, short-term and sudden event: Trading volume movement prediction with graph-based multi-view modeling[EB/OL]. arXiv:2106.03764, 2021.
[19] Murad M M N, et al. WPMixer: Efficient multi-resolution mixing for long-term time series forecasting[C]// Proceedings of the AAAI Conference on Artificial Intelligence. 2025, 39(18): 19581–19588.
[20] Wang S, et al. TimeMixer: Decomposable multiscale mixing for time series forecasting[C]// Proceedings of the International Conference on Learning Representations (ICLR). 2024.
[21] Wang Y, et al. TimeXer: Empowering transformers for time series forecasting with exogenous variables[C]// Advances in Neural Information Processing Systems (NeurIPS). 2024.
[22] Pota M, et al. Multilingual evaluation of pre-processing for BERT-based sentiment analysis of tweets[J]. Expert Systems with Applications, 2021, 181: 115101.
[23] Yin W, Liu Z, Zhang L. Forecasting cryptocurrencies’price with the financial stress index: A graph neural network prediction strategy[EB/OL]. SSRN Preprint, 2022.
[24] Henrique B M, Sobreiro V A, Kimura H. Literature review: Machine learning techniques applied to financial market prediction[J]. Expert Systems with Applications, 2019, 124: 226–251.
[25] Flores B E, Wichern D W. Evaluating forecasts: A look at aggregate bias and accuracy measures[J]. Journal of Forecasting, 2005, 24(6): 433–451.
[26] Li Z, Yang D, Gao B, Jiao Y, Hong Y, Xu T. Multiscale hydrologic applications of the latest satellite precipitation products in the Yangtze River Basin using a distributed hydrologic model[J]. Hydrology and Earth System Sciences, 2015, 16: 407–426.