English
Related papers

Related papers: Multi-modal Spatio-Temporal Transformer for High-r…

200 papers

Traffic forecasting has emerged as a core component of intelligent transportation systems. However, timely accurate traffic forecasting, especially long-term forecasting, still remains an open challenge due to the highly nonlinear and…

Signal Processing · Electrical Eng. & Systems 2021-03-30 Mingxing Xu , Wenrui Dai , Chunmiao Liu , Xing Gao , Weiyao Lin , Guo-Jun Qi , Hongkai Xiong

Using images acquired by different satellite sensors has shown to improve classification performance in the framework of crop mapping from satellite image time series (SITS). Existing state-of-the-art architectures use self-attention…

Computer Vision and Pattern Recognition · Computer Science 2024-06-25 Theresa Follath , David Mickisch , Jan Hemmerling , Stefan Erasmi , Marcel Schwieder , Begüm Demir

To benefit the complementary information between heterogeneous data, we introduce a new Multimodal Transformer (MMFormer) for Remote Sensing (RS) image classification using Hyperspectral Image (HSI) accompanied by another source of data…

Computer Vision and Pattern Recognition · Computer Science 2023-03-24 Bo Zhang , Zuheng Ming , Wei Feng , Yaqian Liu , Liang He , Kaixing Zhao

Currently, in the field of video-text retrieval, there are many transformer-based methods. Most of them usually stack frame features and regrade frames as tokens, then use transformers for video temporal modeling. However, they commonly…

Computer Vision and Pattern Recognition · Computer Science 2024-06-25 Ni Wang , Dongliang Liao , Xing Xu

Spatio-temporal forecasting is crucial in transportation, logistics, and supply chain management. However, current methods struggle with large, complex datasets. We propose a dynamic, multi-modal approach that integrates the strengths of…

Machine Learning · Computer Science 2024-08-27 Sagar Srinivas Sakhinana , Geethan Sannidhi , Chidaksh Ravuru , Venkataramana Runkana

This study proposes a hybrid model based on Transformers, named MSCMHMST, aimed at addressing key challenges in traffic flow prediction. Traditional single-method approaches show limitations in traffic prediction tasks, whereas hybrid…

Machine Learning · Computer Science 2025-03-19 Weiyang Geng , Yiming Pan , Zhecong Xing , Dongyu Liu , Rui Liu , Yuan Zhu

Forecasting the behaviour of complex dynamical systems such as interconnected sensor networks characterized by high-dimensional multivariate time series(MTS) is of paramount importance for making informed decisions and planning for the…

Machine Learning · Computer Science 2024-08-23 Sagar Srinivas Sakhinana , Shivam Gupta , Krishna Sai Sudhir Aripirala , Venkataramana Runkana

Accurate traffic flow prediction is essential for applications like transport logistics but remains challenging due to complex spatio-temporal correlations and non-linear traffic patterns. Existing methods often model spatial and temporal…

Machine Learning · Computer Science 2025-03-18 Jing Chen , Haocheng Ye , Zhian Ying , Yuntao Sun , Wenqiang Xu

Long-term weather forecasting is critical for socioeconomic planning and disaster preparedness. While recent approaches employ finetuning to extend prediction horizons, they remain constrained by the issues of catastrophic forgetting, error…

Computer Vision and Pattern Recognition · Computer Science 2026-05-12 Hao Chen , Tao Han , Jie Zhang , Song Guo , Fenghua Ling , Lei Bai

State space models (SSMs) have shown impressive results on tasks that require modeling long-range dependencies and efficiently scale to long sequences owing to their subquadratic runtime complexity. Originally designed for continuous…

Computation and Language · Computer Science 2023-10-31 Mahan Fathi , Jonathan Pilault , Orhan Firat , Christopher Pal , Pierre-Luc Bacon , Ross Goroshin

The efficient operation of modern cellular networks hinges on the accurate analysis of spatio-temporal traffic data. Mastering these patterns is essential for core network functions, chiefly forecasting future load to pre-empt congestion…

Machine Learning · Computer Science 2026-05-13 Yichen Zhang , Jun Li

Multi-modal Satellite Image Time Series (SITS) analysis faces significant computational challenges for live land monitoring applications. While Transformer architectures excel at capturing temporal dependencies and fusing multi-modal data,…

Image and Video Processing · Electrical Eng. & Systems 2026-03-26 Iris Dumeur , Jérémy Anger , Gabriele Facciolo

Land surface temperature (LST) is vital for land-atmosphere interactions and climate processes. Accurate LST retrieval remains challenging under heterogeneous land cover and extreme atmospheric conditions. Traditional split window (SW)…

Atmospheric and Oceanic Physics · Physics 2025-09-08 Tian Xie , Huanfeng Shen , Menghui Jiang , Juan-Carlos Jiménez-Muñoz , José A. Sobrino , Huifang Li , Chao Zeng

Transformer-based models have emerged as powerful tools for multivariate time series forecasting (MTSF). However, existing Transformer models often fall short of capturing both intricate dependencies across variate and temporal dimensions…

Machine Learning · Computer Science 2024-06-10 Juncheng Liu , Chenghao Liu , Gerald Woo , Yiwei Wang , Bryan Hooi , Caiming Xiong , Doyen Sahoo

Spatio-temporal forecasting is essential for real-world applications such as traffic management and urban computing. Although recent methods have shown improved accuracy, they often fail to account for dynamic deviations between current…

Machine Learning · Computer Science 2025-10-07 Haotian Gao , Zheng Dong , Jiawei Yong , Shintaro Fukushima , Kenjiro Taura , Renhe Jiang

Traditional spatiotemporal models generally rely on task-specific architectures, which limit their generalizability and scalability across diverse tasks due to domain-specific design requirements. In this paper, we introduce…

Computer Vision and Pattern Recognition · Computer Science 2025-03-27 Chen Tang , Xinzhu Ma , Encheng Su , Xiufeng Song , Xiaohong Liu , Wei-Hong Li , Lei Bai , Wanli Ouyang , Xiangyu Yue

Spatio-temporal traffic prediction is crucial in intelligent transportation systems. The key challenge of accurate prediction is how to model the complex spatio-temporal dependencies and adapt to the inherent dynamics in data. Traditional…

Machine Learning · Computer Science 2025-04-15 Wanna Cui , Peizheng Wang , Faliang Yin

Semantic location prediction aims to derive meaningful location insights from multimodal social media posts, offering a more contextual understanding of daily activities than using GPS coordinates. This task faces significant challenges due…

Computer Vision and Pattern Recognition · Computer Science 2024-06-25 Zhizhen Zhang , Ning Wang , Haojie Li , Zhihui Wang

Monitoring the spatiotemporal dynamics of forest CO$_2$ uptake (Gross Primary Production, GPP), remains a central challenge in terrestrial ecosystem research. While Eddy Covariance (EC) towers provide high-frequency estimates, their limited…

Accurately predicting the behavior of complex dynamical systems, characterized by high-dimensional multivariate time series(MTS) in interconnected sensor networks, is crucial for informed decision-making in various applications to minimize…

Machine Learning · Computer Science 2024-08-23 Sagar Srinivas Sakhinana , Krishna Sai Sudhir Aripirala , Shivam Gupta , Venkataramana Runkana