Related papers: Concentration of Additive Functionals for Markov P…
We establish a formula for moments of certain random variables involving positive continuous additive functionals (PCAFs) of standard processes which have absolutely continuous transition functions and are in duality with standard processes…
The recent study by B. De Bruyne, S. N. Majumdar, H. Orland and G. Schehr [arXiv:2110.07573], concerning the conditioning of the Brownian motion and of random walks on global dynamical constraints over a finite time-window $T$, is…
We obtain necessary and sufficient conditions for the regular variation of the variance of partial sums of functionals of discrete and continuous-time stationary Markov processes with normal transition operators. We also construct a class…
We establish verifiable general sufficient conditions for exponential or subexponential ergodicity of Markov processes that may lack the strong Feller property. We apply the obtained results to show exponential ergodicity of a variety of…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
Using a martingale concentration inequality, concentration bounds `from time $n_0$ on' are derived for stochastic approximation algorithms with contractive maps and both martingale difference and Markov noises. These are applied to…
This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…
We prove an apparently novel concentration of measure result for Markov tree processes. The bound we derive reduces to the known bounds for Markov processes when the tree is a chain, thus strictly generalizing the known Markov process…
We provide necessary and sufficient conditions for convergence of exponential integrals of Markov additive processes. Other than in the classical L\'evy case studied by Erickson and Maller we have to distinguish between almost sure…
We study the performance of a stochastic algorithm based on the power method that adaptively learns the large deviation functions characterizing the fluctuations of additive functionals of Markov processes, used in physics to model…
A variational formula for the asymptotic variance of general Markov processes is obtained. As application, we get a upper bound of the mean exit time of reversible Markov processes, and some comparison theorems between the reversible and…
We solve two problems related to the fluctuations of time-integrated functionals of Markov diffusions, used in physics to model nonequilibrium systems. In the first we derive and illustrate the appropriate boundary conditions on the…
In this paper, we investigate the concentration properties of cumulative reward in Markov Decision Processes (MDPs), focusing on both asymptotic and non-asymptotic settings. We introduce a unified approach to characterize reward…
We present a multivariate central limit theorem for a general class of interacting Markov chain Monte Carlo algorithms used to solve nonlinear measure-valued equations. These algorithms generate stochastic processes which belong to the…
We derive a moment formula for generalized fractional polynomial processes, i.e., for polynomial-preserving Markov processes time-changed by an inverse L\'evy-subordinator. If the time change is inverse $\alpha$-stable, the time-derivative…
We prove a functional limit theorem for Markov chains that, in each step, move up or down by a possibly state dependent constant with probability $1/2$, respectively. The theorem entails that the law of every one-dimensional regular…
We obtain functional central limit theorems for both discrete time expressions of the form $1/\sqrt{N}\sum_{n=1}^{[Nt]}(F(X(q_1(n)),\ldots, X(q_{\ell}(n)))-\bar{F})$ and similar expressions in the continuous time where the sum is replaced…
We consider Markov processes in continuous time with state space $\posint^N$ and provide two sufficient conditions and one necessary condition for the existence of moments $E(\|X(t)\|^r)$ of all orders $r \in \nat$ for all $t \geq 0$. The…
We revisit affine diffusion processes on general and on the canonical state space in particular. A detailed study of theoretic and applied aspects of this class of Markov processes is given. In particular, we derive admissibility conditions…
Focusing on stochastic systems arising in mean-field models, the systems under consideration belong to the class of switching diffusions, in which continuous dynamics and discrete events coexist and interact. The discrete events are modeled…