Related papers: A central limit theorem for temporally non-homogen…
This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying…
In this paper, a very useful lemma (in two versions) is proved: it simplifies notably the essential step to establish a Lindeberg central limit theorem for dependent processes. Then, applying this lemma to weakly dependent processes…
In this paper we investigate a sequence of square integrable random processes with space varying memory. We establish sufficient conditions for the central limit theorem in the space $L^2(\mu)$ for the partial sums of the sequence of random…
We prove a central limit theorem for a sequence of random variables whose means are ambiguous and vary in an unstructured way. Their joint distribution is described by a set of measures. The limit is (not the normal distribution and is)…
In this paper, under mild assumptions, we derive a law of large numbers, a central limit theorem with an error estimate, an almost sure invariance principle and a variant of Chernoff bound in finite-state hidden Markov models. These limit…
In this paper we investigate the local limit theorem for additive functionals of a nonstationary Markov chain with finite or infinite second moment. The moment conditions are imposed on the individual summands and the weak dependence…
The Dobrushin comparison theorem is a powerful tool to bound the difference between the marginals of high-dimensional probability distributions in terms of their local specifications. Originally introduced to prove uniqueness and decay of…
The Master equation on directed networks - also called the differential Chapman-Kolmogorov equation - is a linear differential equation, which describes the probability evolution in a discrete system. While this is well understood, if the…
Fleming-Viot type particle systems represent a classical way to approximate the distribution of a Markov process with killing, given that it is still alive at a final deterministic time. In this context, each particle evolves independently…
We prove the existence of limiting distributions for a large class of Markov chains on a general state space in a random environment. We assume suitable versions of the standard drift and minorization conditions. In particular, the system…
Suppose $B_i:= B(p,r_i)$ are nested balls of radius $r_i$ about a point $p$ in a dynamical system $(T,X,\mu)$. The question of whether $T^i x\in B_i$ infinitely often (i. o.) for $\mu$ a.e.\ $x$ is often called the shrinking target problem.…
We study the problem of finite-horizon probabilistic invariance for discrete-time Markov processes over general (uncountable) state spaces. We compute discrete-time, finite-state Markov chains as formal abstractions of general Markov…
In this paper, we propose two discontinuous dynamical systems in continuous time with guaranteed prescribed finite-time local convergence to strict local minima of a given cost function. Our approach consists of exploiting a Lyapunov-based…
We study an infinite horizon optimal stopping problem which arises naturally in the optimal timing of a firm/project sale or in the valuation of natural resources: the functional to be maximised is a sum of a discounted running reward and a…
This paper will provide several classes of strictly stationary, countable-state, irreducible, aperiodic Markov chains that are reversible and have finite second moments, such that the central limit theorem fails to hold. The main purpose is…
We compute some dependence coefficients for the stationary Markov chain whose transition kernel is the Perron-Frobenius operator of an expanding map $T$ of $[0, 1]$ with a neutral fixed point. We use these coefficients to prove a central…
This paper considers the infinite horizon optimal control problem for nonlinear systems. Under the condition of nonlinear controllability of the system to any terminal set containing the origin and forward invariance of the terminal set, we…
Two time scale stochastic approximation algorithms emulate singularly perturbed deterministic differential equations in a certain limiting sense, i.e., the interpolated iterates on each time scale approach certain differential equations in…
Imprecise continuous-time Markov chains are a robust type of continuous-time Markov chains that allow for partially specified time-dependent parameters. Computing inferences for them requires the solution of a non-linear differential…
We propose a vector linear programming formulation for a non-stationary, finite-horizon Markov decision process with vector-valued rewards. Pareto efficient policies are shown to correspond to efficient solutions of the linear program, and…