Related papers: On Function of Evolution of Distribution for Time …
In many applications, for example when computing statistics of fast subsystems in a multiscale setting, we wish to find the stationary distributions of systems of continuous time Markov chains. Here we present a class of models that appears…
This article describes an accurate procedure for computing the mean first passage times of a finite irreducible Markov chain and a Markov renewal process. The method is a refinement to the Kohlas, Zeit fur Oper Res, 30,197-207, (1986)…
The modeling of natural phenomena via a Markov process --- a process for which the future is independent of the past, given the present--- is ubiquitous in many fields of science. Within this context, it is of foremost importance to develop…
We focus on variational inference in dynamical systems where the discrete time transition function (or evolution rule) is modelled by a Gaussian process. The dominant approach so far has been to use a factorised posterior distribution,…
Motivated by queueing systems with heterogeneous parallel servers, we consider a class of structured multi-dimensional Markov processes whose state space can be partitioned into two parts: a finite set of boundary states and a structured…
A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…
We show the variational convergence of an irreversible Markov jump process describing a finite stochastic particle system to the solution of a countable infinite system of deterministic time-inhomogeneous quadratic differential equations…
A heterogeneous continuous time random walk is an analytical formalism for studying and modeling diffusion processes in heterogeneous structures on microscopic and macroscopic scales. In this paper we study both analytically and numerically…
We consider an interacting particle Markov process for Darwinian evolution in an asexual population with non-constant population size, involving a linear birth rate, a density-dependent logistic death rate, and a probability $\mu$ of…
Dropout represents a typical issue to be addressed when dealing with longitudinal studies. If the mechanism leading to missing information is non-ignorable, inference based on the observed data only may be severely biased. A frequent…
In the last years, many authors studied a class of continuous time semi-Markov processes obtained by time-changing Markov processes by hitting times of independent subordinators. Such processes are governed by integro-differential…
The usual random walk on a group (homogeneous both in time and in space) is determined by a probability measure on the group. In a random walk with random transition probabilities this single measure is replaced with a stationary sequence…
Reinforcement learning algorithms are typically designed for generic Markov Decision Processes (MDPs), where any state-action pair can lead to an arbitrary transition distribution. In many practical systems, however, only a subset of the…
Mean-field models are a popular tool in a variety of fields. They provide an understanding of the impact of interactions among a large number of particles or people or other "self-interested agents", and are an increasingly popular tool in…
In this paper, we investigate the distributed convex optimization problem over a multi-agent system with Markovian switching communication networks. The objective function is the sum of each agent's local objective function, which cannot be…
The time reversal of a completely-positive, nonequilibrium discrete-time quantum Markov evolution is derived via a suitable adjointness relation. Space-time harmonic processes are introduced for the forward and reverse-time transition…
The time evolution of a bounded quantum system is considered in the framework of the orthogonal, unitary and symplectic circular ensembles of random matrix theory. For an $N$ dimensional Hilbert space we prove that in the large $N$ limit…
We solve a simple model that supports a dynamic phase transition and show conditions for the existence of the transition. Using methods of large deviation theory we analytically compute the probability distribution for activity and entropy…
We develop a Markov process viewpoint for discrete circular distributions motivated by directional-statistics settings where angles are observed on a finite grid and evolve over time. On the $m$-point discrete circle, the cycle graph, we…
We study a one-dimensional Markov modulated random walk with jumps. It is assumed that amplitudes of jumps as well as a chosen velocity regime are random and depend on a time spent by the process at a previous state of the underlying Markov…