Related papers: Regularity of harmonic functions for some Markov c…
We consider two types of discrete-time Markov chains where the state space is a graded poset and the transitions are taken along the covering relations in the poset. The first type of Markov chain goes only in one direction, either up or…
We consider continuous-space, discrete-time Markov chains on $\mathbb{R}^d$, that admit a finite number $N$ of metastable states. Our main motivation for investigating these processes is to analyse random Poincar\'e maps, which describe…
We consider infinite harmonic chain on the real line with deterministic dynamics (no stochasticity). We indicate classes of uniformly bounded initial conditions when the trajectories of particles stay uniformly bounded.
In the article the distributions of overjump functionals for almost semi-continuous processes on a finite irreducible Markov chain are considered.
In this study, a new extension of the Markov Renewal theory is introduced by allowing time to evolve in multiple dimensions. The resulting chains are referred to as multi-time Markov Renewal chains and since this extension is new, the state…
We consider the computational task of sampling a bit string $x$ from a distribution $\pi(x)=|\langle x|\psi\rangle|^2$, where $\psi$ is the unique ground state of a local Hamiltonian $H$. Our main result describes a direct link between the…
The equivalence of regularity of a Q-matrix with its bounded perturbations is proved and a integration by parts formula is established for the associated Feller minimal transition functions.
We apply the method of differential inequalities for the computation of upper bounds for the rate of convergence to the limiting regime for one specific class of (in)homogeneous continuous-time Markov chains. To obtain these estimates, we…
We study time-inhomogeneous Markov chains to obtain quantitative results on their asymptotic behavior. We use Poincar\'e, Nash, and logarithmic-Sobolev inequalities. We assume that our Markov chain admits a finite invariant measure at each…
We consider a discrete-time, continuous-state random walk with steps uniformly distributed in a disk of radius of $h$. For a simply connected domain $D$ in the plane, let $\omega_h(0,\cdot;D)$ be the discrete harmonic measure at $0\in D$…
We construct and study branching Markov processes on the space of finite configurations of the state space of a given standard process, controlled by a branching kernel and a killing one. In particular, we may start with a superprocess,…
We study a variable length Markov chain model associated with a group of stationary processes that share the same context tree but each process has potentially different conditional probabilities. We propose a new model selection and…
This paper focuses on a class of continuous-time controlled Markov chains with time-inconsistent and distribution-dependent cost functional (in some appropriate sense). A new definition of time-inconsistent distribution-dependent…
We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this…
We consider moments of the return times (or first hitting times) in a discrete time discrete space Markov chain. It is classical that the finiteness of the first moment of a return time of one state implies the finiteness of the first…
In some papers on infinite Markov chains in $\mathbb{Z}^d$, and notably in the work of R.A. Minlos and collaborators, one can prove the existence of a spectral gap for a suitable subspace of local functions. We consider functions of the…
We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterise the value function via HJB equation…
We study certain properties of the function space of autocorrelation functions of Unit Continuous Time Markov Chains (CTMCs). It is shown that under particular conditions, the $L^p$ norm of the autocorrelation function of arbitrary finite…
Continuity equations associated to continuous-time Markov processes can be considered as Euclidean Schr\"odinger equations, where the non-hermitian quantum Hamiltonian $\bold{H}={\bold{div}}{\bold J}$ is naturally factorized into the…
We review recent results on the metastable behavior of continuous-time Markov chains derived through the characterization of Markov chains as unique solutions of martingale problems.