Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting

Wan Zhang; Qinjie Lin; Chan Lee; Weijian Li; Han Liu; Kai Zhang

Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting

Machine Learning 2026-07-24 v1 Machine Learning

Authors: Wan Zhang , Qinjie Lin , Chan Lee , Weijian Li , Han Liu , Kai Zhang

Abstract

Forecasting multiple time-series with high-dimensional covariates presents a core challenge: unifying common temporal patterns while retaining meaningful series-specific information. We introduce Hopformer (Homogeneity-Pursuit Transformer), a two-stage framework that addresses this challenge. In the first stage, we perform a Sparsity Pattern Aggregation (SPA) scheme extracting a common low-variance trend that incorporates the covariates. This acts as a homogenization layer. In the second stage, a LoRA-fine-tuned Transformer models the remaining complex dependencies in the residual. Our method is theoretically grounded. We prove that SPA achieves a near-optimal bias-variance trade-off via an oracle inequality. We also provide generalization bounds for the second stage under dependent time series data. Hopformer sets a new state of the art, improving MASE by an average of 6.56% across synthetic and real-world forecasting benchmarks.

Keywords

time series forecasting transformer homogeneity pursuit sparsity pattern aggregation LoRA multi-series forecasting

Cite

@article{arxiv.2607.22299,
  title  = {Hopformer: Homogeneity-Pursuit Transformer for Time Series Forecasting},
  author = {Wan Zhang and Qinjie Lin and Chan Lee and Weijian Li and Han Liu and Kai Zhang},
  journal= {arXiv preprint arXiv:2607.22299},
  year   = {2026}
}