Related papers: Markov processes on a circular lattice
The evolution of a continuous time Markov process with a finite number of states is usually calculated by the Master equation - a linear differential equations with a singular generator matrix. We derive a general method for reducing the…
In many applications, for example when computing statistics of fast subsystems in a multiscale setting, we wish to find the stationary distributions of systems of continuous time Markov chains. Here we present a class of models that appears…
We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…
The number of observable degrees of freedom is typically limited in experiments. Here, we consider discrete Markov networks in which an observer has access to a few visible transitions and the waiting times between these transitions.…
A general theory is developed to study individual based models which are discrete in time. We begin by constructing a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are…
Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…
We introduce a four-parameter extended family of distributions related to the wrapped Cauchy distribution on the circle. The proposed family can be derived by altering the settings of a problem in Brownian motion which generates the wrapped…
Density dependent families of Markov chains, such as the stochastic models of mass-action chemical kinetics, converge for large values of the indexing parameter $N$ to deterministic systems of differential equations (Kurtz, 1970). Moreover…
A diffusion taking value in probability measures on a graph with a vertex set $V$, $\sum_{i\in V}x_i\delta_i$, is studied. The masses on each vertices satisfy the stochastic differential equation of the form $dx_i=\sum_{j\in…
We study the Markov chain on $\mathbf{F}_p$ obtained by applying a function $f$ and adding $\pm\gamma$ with equal probability. When $f$ is a linear function, this is the well-studied Chung--Diaconis--Graham process. We consider two cases:…
We propose a Markov chain simulation method to generate simple connected random graphs with a specified degree sequence and level of clustering. The networks generated by our algorithm are random in all other respects and can thus serve as…
Discrete-time discrete-state finite Markov chains are versatile mathematical models for a wide range of real-life stochastic processes. One of most common tasks in studies of Markov chains is computation of the stationary distribution.…
We study Markov processes where the "time" parameter is replaced by paths in a directed graph from an initial vertex to a terminal one. Along each directed path the process is Markov and has the same distribution as the one along any other…
A study of time homogeneous, real valued Markov processes with a special property and a non-atomic initial distribution is provided. The new notion of a function of evolution of distribution which determines the dependency between one…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
The success of generative modeling in continuous domains has led to a surge of interest in generating discrete data such as molecules, source code, and graphs. However, construction histories for these discrete objects are typically not…
We consider a discrete-time dynamical process on graphs, firstly introduced in connection with a protocol for controlling large networks of spin 1/2 quantum mechanical particles [Phys. Rev. Lett. 99, 100501 (2007)]. A description is as…
Strongly Rayleigh distributions are natural generalizations of product and determinantal probability distributions and satisfy strongest form of negative dependence properties. We show that the "natural" Monte Carlo Markov Chain (MCMC) is…
We derive and analyze new diffusion approximations of stationary distributions of Markov chains that are based on second- and higher-order terms in the expansion of the Markov chain generator. Our approximations achieve a higher degree of…
Suppose X is a multivariate diffusion process that is observed discretely in time. At each observation time, a transformation of the state of the process is observed with noise. The smoothing problem consists of recovering the path of the…