相关论文: Practical Criterion for Uniqueness of $Q$-processe…
Verification of infinite-state Markov chains is still a challenge despite several fruitful numerical or statistical approaches. For decisive Markov chains, there is a simple numerical algorithm that frames the reachability probability as…
For Markov processes with absorption, we provide general criteria ensuring the existence and the exponential non-uniform convergence in total variation norm to a quasi-stationary distribution. We also characterize a subset of its domain of…
We derive a sufficient condition for a $k$-th order homogeneous Markov chain $\mathbf{Z}$ with finite alphabet $\mathcal{Z}$ to have a unique invariant distribution on $\mathcal{Z}^k$. Specifically, let $\mathbf{X}$ be a first-order,…
We consider moments of the return times (or first hitting times) in a discrete time discrete space Markov chain. It is classical that the finiteness of the first moment of a return time of one state implies the finiteness of the first…
We provide a sufficient condition for the continuity of real valued permanental processes. When applied to the subclass of permanental processes which consists of squares of Gaussian processes, we obtain the sufficient condition for…
We construct a non-decreasing pure jump Markov process, whose jump measure heavily depends on the values taken by the process. We determine the singularity spectrum of this process, which turns out to be random and to depend locally on the…
Starting from a Markov chain with a finite alphabet, we consider the chain obtained when all but one symbol are undistinguishable for the practitioner. We study necessary and sufficient conditions for this chain to have continuous…
We derive a necessary and sufficient condition for a quantum process to be Markovian which coincides with the classical one in the relevant limit. Our condition unifies all previously known definitions for quantum Markov processes by…
We study existence, uniqueness and computability of solutions for a class of discrete time recursive utilities models. By combining two streams of the recent literature on recursive preferences---one that analyzes principal eigenvalues of…
This paper aims to develop the stability theory for singular stochastic Markov jump systems with state-dependent noise, including both continuous- and discrete-time cases. The sufficient conditions for the existence and uniqueness of a…
We prove a central limit theorem for a class of additive processes that arise naturally in the theory of finite horizon Markov decision problems. The main theorem generalizes a classic result of Dobrushin (1956) for temporally…
We consider Markov processes in continuous time with state space $\posint^N$ and provide two sufficient conditions and one necessary condition for the existence of moments $E(\|X(t)\|^r)$ of all orders $r \in \nat$ for all $t \geq 0$. The…
This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…
This paper focuses on optimizing probabilities of events of interest defined over general controlled discrete-time Markov processes. It is shown that the optimization over a wide class of $\omega$-regular properties can be reduced to the…
The cutoff phenomenon is an abrupt transition from out of equilibrium to equilibrium undergone by certain Markov processes in the limit where the size of the state space tends to infinity: instead of decaying gradually over time, their…
More than a century after the inception of quantum theory, the question of which traits and phenomena are fundamentally quantum remains under debate. Here we give an answer to this question for temporal processes which are probed…
We formally extend the notion of Markov order to open quantum processes by accounting for the instruments used to probe the system of interest at different times. Our description recovers the classical Markov order property in the…
We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of…
We establish sufficient conditions for exponential convergence to a unique quasi-stationary distribution in the total variation norm. These conditions also ensure the existence and exponential ergodicity of the Q-process, the process…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…