English

Towards Continuous-time Causal Foundation Models

Machine Learning 2026-05-29 v1 Data Analysis, Statistics and Probability Methodology

Abstract

Extending discrete-time causal Prior-data Fitted Networks for time series to continuous time invites writing the mechanism as a stochastic differential equation (SDE) -- but if the SDE is integrated \emph{once per observation gap}, the trajectory law depends on when it is observed, and the prior remains a discrete-time Markov model in SDE clothing. We propose a precise continuity criterion -- trajectory-law invariance to the observation schedule -- together with a three-tier taxonomy (discrete; naive observation-grid integration; fine-grid integration with decoupled observation) and a construction realising the top tier on a random DAG with OU or small-MLP nonlinear drifts, irregular observation schedules, and hard / soft / time-varying interventions. A 2×22 \times 2 encoder ×\times integrator ablation, run independently on a linear and a nonlinear prior, finds fine-grid integration beats naive on 8/8 cells (sign-consistency p<1/256p < 1/256) with the gap growing as the eval grid refines; the encoder axis is null with fine integration but time-aware-leading with naive. We release the prior and a preliminary zero-shot protocol on pharmacokinetic and physical-system data.

Keywords

Cite

@article{arxiv.2605.28880,
  title  = {Towards Continuous-time Causal Foundation Models},
  author = {Dennis Thumm and Ruben Wiedemann and Ying Chen},
  journal= {arXiv preprint arXiv:2605.28880},
  year   = {2026}
}

Comments

ICML 2026 2nd Workshop on Foundation Models for Structured Data (FMSD)