Nonequilibrium steady-state dynamics of Markov processes on graphs

We propose an analytic approach for the steady-state dynamics of Markov processes on locally tree-like graphs. It is based on time-translation invariant probability distributions for edge trajectories, which we encode in terms of infinite matrix products. For homogeneous ensembles on regular graphs, the distribution is parametrized by a single $d\times d\times r^2$ tensor, where $r$ is the number of states per variable, and $d$ is the matrix-product bond dimension. While the method becomes exact in the large-$d$ limit, it typically provides highly accurate results even for small bond dimensions $d$. The $d^2r^2$ parameters are determined by solving a fixed point equation, for which we provide an efficient belief-propagation procedure. We apply this approach to a variety of models, including Ising-Glauber dynamics with symmetric and asymmetric couplings, as well as the SIS model. Even for small $d$, the results are compatible with Monte Carlo estimates and accurately reproduce known exact solutions. The method provides access to precise temporal correlations, which, in some regimes, would be virtually impossible to estimate by sampling.