English

Onsager-Machlup Posterior Transport for Deep Gaussian Processes

Machine Learning 2026-05-25 v1

Abstract

Approximate inference over inducing variables is the central computational bottleneck of Deep Gaussian Processes (DGPs). Existing methods either fit an explicit density qϕ(\bU)q_\phi(\bU) by an ELBO (DSVI, IPVI, DDVI, DBVI) or sample by MCMC (SGHMC). We instead frame DGP inference as \emph{posterior transport}: learn a deterministic sampler that maps a tractable reference measure to posterior-relevant inducing variables, regularised by a path prior derived from the Doob-bridged reference diffusion. Our realisation, \textbf{OM-Path} (formally FBVI-bridge-Path), uses Song's probability-flow ODE applied to DBVI's Doob-bridged forward SDE; the reference drift is closed-form from the bridge marginal coefficients (no score matching) and the path regulariser is the \textbf{Onsager--Machlup action}. At the finite-ϵ\epsilon value used at training, the objective is the negative log unnormalised density of a tempered Doob-bridge path posterior, and Theorem 1 identifies it with the same posterior's small-noise MAP path via the Freidlin--Wentzell LDP. Two strict path-space ELBO variants on the same bridge backbone (FFJORD log-det; OM-regularised CNF) are derived as ablations. Under a matched-seed paired Wilcoxon test against DBVI on seven UCI regression benchmarks, OM-Path delivers statistically significant wins on the two largest datasets (\textit{power}: p ⁣= ⁣0.014p\!=\!0.014, NLL 0.012\mathbf{0.012} matching the DSVI baseline of 0.0170.017; \textit{protein}: p ⁣= ⁣0.002p\!=\!0.002, RMSE 0.716\mathbf{0.716} vs.\ 0.7640.764, NLL 1.086\mathbf{1.086} vs.\ 1.1491.149), statistical ties on \textit{yacht} / \textit{qsar}, and concedes \textit{boston} / \textit{energy} / \textit{concrete} to DBVI on small-NN noisy data. The strict-ELBO variants do not clear DBVI on any UCI metric: in this regime, reducing the variance of the path objective dominates exact-density tracking.

Cite

@article{arxiv.2605.23434,
  title  = {Onsager-Machlup Posterior Transport for Deep Gaussian Processes},
  author = {Jian Xu and Delu Zeng and John Paisley and Qibin Zhao},
  journal= {arXiv preprint arXiv:2605.23434},
  year   = {2026}
}
R2 v1 2026-07-22T07:27:58.134Z