Related papers: A Compositional Framework for Markov Processes
This paper introduces Discrete Markov Probabilistic Models (DMPMs), a novel discrete diffusion algorithm for discrete data generation. The algorithm operates in discrete bit space, where the noising process is a continuous-time Markov chain…
The goal of this work is to formally abstract a Markov process evolving in discrete time over a general state space as a finite-state Markov chain, with the objective of precisely approximating its state probability distribution in time,…
Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain…
We consider a stochastic process which is (a) described by a continuous-time Markov chain on only short time-scales and (b) constrained to conserve a number of hidden quantities on long time-scales. We assume that the transition matrix of…
In systems of programmable matter, we are given a collection of simple computation elements (or particles) with limited (constant-size) memory. We are interested in when they can self-organize to solve system-wide problems of movement,…
This paper is concerned with a compositional approach for constructing finite Markov decision processes of interconnected discrete-time stochastic control systems. The proposed approach leverages the interconnection topology and a notion of…
A continuous-time Markov process $X$ can be conditioned to be in a given state at a fixed time $T > 0$ using Doob's $h$-transform. This transform requires the typically intractable transition density of $X$. The effect of the $h$-transform…
The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…
We explore the concept of a consistent exchangeable survival process - a joint distribution of survival times in which the risk set evolves as a continuous-time Markov process with homogeneous transition rates. We show a correspondence with…
Continuous-time Markov chains describing interacting processes exhibit a state space that grows exponentially in the number of processes. This state-space explosion renders the computation or storage of the time-marginal distribution, which…
A branching process in a Markovian environment consists of an irreducible Markov chain on a set of "environments" together with an offspring distribution for each environment. At each time step the chain transitions to a new random…
For a wide class of continuous-time Markov processes, including all irreducible hypoelliptic diffusions evolving on an open, connected subset of $\RL^d$, the following are shown to be equivalent: (i) The process satisfies (a slightly weaker…
Changing time of simple continuous-time Markov counting processes by independent unit-rate Poisson processes results in Markov counting processes for which we provide closed-form transition rates via composition of trajectories and with…
Time-reversal symmetry of microscopic laws dictates that the equilibrium distribution of a stochastic process must obey the detailed balance. On the other hand, cyclic Markov processes that do not admit equilibrium distributions with…
The seek for a new universal formulation for describing various non-equilibrium processes is a central task of modern non-equilibrium thermodynamics. In this paper, a novel steady-state thermodynamic formalism was established for general…
We develop a Markov process viewpoint for discrete circular distributions motivated by directional-statistics settings where angles are observed on a finite grid and evolve over time. On the $m$-point discrete circle, the cycle graph, we…
A general theory is developed to study individual based models which are discrete in time. We begin by constructing a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are…
Even simply-defined, finite-state generators produce stochastic processes that require tracking an uncountable infinity of probabilistic features for optimal prediction. For processes generated by hidden Markov chains the consequences are…
We consider the Fluctuation Dissipation Theorem (FDT) of statistical physics from a mathematical perspective. We formalize the concept of "linear response function" in the general framework of Markov processes. We show that for processes…
A method of constructing Markov chains on finite state spaces is provided. The chain is specified by three constraints: stationarity, dependence and marginal distributions. The generalized Pythagorean theorem in information geometry plays a…