Related papers: Random-step Markov processes
Consider a stochastic process $\{X(t)\}$ on a finite state space $ {\sf X}=\{1,\dots, d\}$. It is conditionally Markov, given a real-valued `input process' $\{\zeta(t)\}$. This is assumed to be small, which is modeled through the scaling,…
We study an $n$-species $t$-PushTASEP, an integrable long-range stochastic process, on a one-dimensional periodic lattice with inhomogeneities $x_1,\ldots,x_L$ and arbitrary capacity $l$ at each lattice site. The Markov matrix is identified…
Markov processes with stochastic resetting towards the origin generically converge towards non-equilibrium steady-states. Long dynamical trajectories can be thus analyzed via the large deviations at Level 2.5 for the joint probability of…
We revisit processes generated by iterated random functions driven by a stationary and ergodic sequence. Such a process is called strongly stable if a random initialization exists, for which the process is stationary and ergodic, and for…
Empirical processes for stationary, causal sequences are considered. We establish empirical central limit theorems for classes of indicators of left half lines, absolutely continuous functions and piecewise differentiable functions. Sample…
The slow processes of metastable stochastic dynamical systems are difficult to access by direct numerical simulation due the sampling problem. Here, we suggest an approach for modeling the slow parts of Markov processes by approximating the…
We study a class of Markov processes with finite state space and continuous time that have product form stationary distributions. We obtain a number of examples that can generate conjectures for diffusions with inert drift.
We consider a general piecewise deterministic Markov process (PDMP) $X=\{X_t\}_{t\geqslant 0}$ with measure-valued generator $\mathcal{A}$, for which the conditional distribution function of the inter-occurrence time is not necessarily…
In this paper, we consider semi-Markov processes whose transition times and transition probabilities depend on a small parameter $\varepsilon$. Understanding the asymptotic behavior of such processes is needed in order to study the…
We propose the entropy of random Markov trajectories originating and terminating at a state as a measure of the stability of a state of a Markov process. These entropies can be computed in terms of the entropy rates and stationary…
Linear fractional Galton-Watson branching processes in i.i.d.~random environment are, on the quenched level, intimately connected to random difference equations by the evolution of the random parameters of their linear fractional marginals.…
The methodology based on the random walk processes is adapted and applied to a comprehensive analysis of the statistical properties of the probability fluxes. To this aim we define a simple model of the Markovian stochastic dynamics on a…
We obtain the posterior distribution of a random process conditioned on observing the empirical frequencies of a finite sample path. We find under a rather broad assumption on the "dependence structure" of the process, {\em c.f.}…
It is well-known that 0 is the absorbing state for a branching system. Each particle in the system lives a random long time and gives a random number of new particles at its death time. It stops when the system has no particle. This paper…
This paper presents a general approach to linear stochastic processes driven by various random noises. Mathematically, such processes are described by linear stochastic differential equations of arbitrary order (the simplest non-trivial…
The aim of the paper is to understand how the inclusion of more and more time-scales into a stochastic stationary Markovian process affects its conditional probability. To this end, we consider two Gaussian processes: (i) a short-range…
The paper deals with a certain class of random evolutions. We develop a construction that yields an invariant measure for a continuous-time Markov process with random transitions. The approach is based on a particular way of constructing…
Markov decision processes (MDP) are a well-established model for sequential decision-making in the presence of probabilities. In robust MDP (RMDP), every action is associated with an uncertainty set of probability distributions, modelling…
In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…
We define a de Bruijn process with parameters n and L as a certain continuous-time Markov chain on the de Bruijn graph with words of length L over an n-letter alphabet as vertices. We determine explicitly its steady state distribution and…