CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Data
Abstract
Time series foundation models (TSFMs) have recently gained significant attention due to their strong zero-shot capabilities and widespread real-world applications. Such models typically require a computationally costly pre-training on large-scale, carefully curated collections of real-world sequences. To allow for a sample-efficient pre-training of TSFMs, we propose \textsc{CauKer}, a novel algorithm designed to generate diverse, causally coherent synthetic time series with realistic trends, seasonality, and nonlinear interactions. \textsc{CauKer} combines Gaussian Process (GP) kernel composition with Structural Causal Models (SCM) to produce data for sample-efficient pre-training of state-of-the-art classification TSFMs having different architectures and following different pre-training approaches. Additionally, our experiments reveal that \textsc{CauKer}-generated datasets exhibit clear scaling laws for both dataset size (10K to 10M samples) and model capacity (1M to 783M parameters), unlike real-world datasets, which display irregular scaling behavior. The source code is publicly available at https://github.com/ShifengXIE/CauKer.
Cite
@article{arxiv.2508.02879,
title = {CauKer: Classification Time Series Foundation Models Can Be Pretrained on Synthetic Data},
author = {Shifeng Xie and Vasilii Feofanov and Ambroise Odonnat and Lei Zan and Marius Alonso and Jianfeng Zhang and Themis Palpanas and Lujia Pan and Keli Zhang and Ievgen Redko},
journal= {arXiv preprint arXiv:2508.02879},
year = {2026}
}
Comments
This manuscript combines material from the ICML 2025 TSFM Workshop paper and the ICLR 2026 Main Track paper