Related papers: An Elementary Derivation of the Large Deviation Ra…

Consider the normalized partial sums of a real-valued function $F$ of a Markov chain, \[\phi_n:=n^{-1}\sum_{k=0}^{n-1}F(\Phi(k)),\qquad n\ge1.\] The chain $\{\Phi(k):k\ge0\}$ takes values in a general state space $\mathsf {X}$, with…

Let (X_n,Y_n) be i.i.d. random vectors. Let W(x) be the partial sum of Y_n just before that of X_n exceeds x>0. Motivated by stochastic models for neural activity, uniform convergence of the form $\sup_{c\in I}|a(c,x)\operatorname…

We present and explore a general method for deriving a Lie-Markov model from a finite semigroup. If the degree of the semigroup is $k$, the resulting model is a continuous-time Markov chain on $k$ states and, as a consequence of the product…

The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…

A rigorous connection between large deviations theory and Gamma-convergence is established. Applications include representations formulas for rate functions, a contraction principle for measurable maps, a large deviations principle for…

We consider finite-dimensional systems of linear stochastic differential equations ${\partial_t}{x_k}\left( t \right) = {A_{kp}}\left( t \right){x_p}\left( t \right)$, ${\bf A}(t)$ being a stationary continuous statistically isotropic…

In this paper, we consider the finite-state approximation of a discrete-time constrained Markov decision process (MDP) under the discounted and average cost criteria. Using the linear programming formulation of the constrained discounted…

We consider large deviations of empirical measures of diffusion processes. In a first part, we present conditions to obtain a large deviations principle (LDP) for a precise class of unbounded functions. This provides an analogue to the…

We study a novel large dimensional approximate factor model with regime changes in the loadings driven by a latent first order Markov process. By exploiting the equivalent linear representation of the model, we first recover the latent…

We consider the problem of controlling a Markov decision process (MDP) with a large state space, so as to minimize average cost. Since it is intractable to compete with the optimal policy for large scale problems, we pursue the more modest…

In this paper, we study the finite-sample statistical rates of distributional temporal difference (TD) learning with linear function approximation. The aim of distributional TD learning is to estimate the return distribution of a discounted…

We prove a Large Deviation Principle for Piecewise Deterministic Markov Processes (PDMPs). This is an asymptotic estimate for the probability of a trajectory in the large size limit. Explicit Euler-Lagrange equations are determined for…

This paper describes a model for generating time series which exhibit the statistical phenomenon known as long-range dependence (LRD). A Markov Modulated Process based upon an infinite Markov chain is described. The work described is…

In this paper, we provide a unified analysis of temporal difference learning algorithms with linear function approximators by exploiting their connections to Markov jump linear systems (MJLS). We tailor the MJLS theory developed in the…

We consider a pure jump process $\{X_t\}_{t\ge 0}$ with values in a finite state space $S= \{1, \ldots, d\}$ for which the jump rates at time instant $t$ depend on the occupation measure $L_t \doteq t^{-1} \int_0^t \delta_{X_s}\,ds$. Such…

Let $\Sigma_{A}(\mathbb{N})$ be a topologically mixing countable Markov shift with the BIP property over the alphabet $\mathbb{N}$ and $f: \Sigma_{A}(\mathbb{N}) \rightarrow \mathbb{R}$ a potential satisfying the Walters condition with…

We investigate the large deviation principle (LDP) of the stationary solutions of stochastic functional differential equations (SFDEs) with infinite delay under small random perturbation. First, we demonstrate the existence and uniqueness…

Let $S_N$ be the sum of vector-valued functions defined on a finite Markov chain. An analogue of the Bernstein--Hoeffding inequality is derived for the probability of large deviations of $S_N$ and relates the probability to the spectral gap…

The $W$-random graphs provide a flexible framework for modeling large random networks. Using the Large Deviation Principle (LDP) for $W$-random graphs from [9], we prove the LDP for the corresponding class of random symmetric…

Particle approximations for certain nonlinear and nonlocal reaction-diffusion equations are studied using a system of Brownian motions with killing. The system is described by a collection of i.i.d. Brownian particles where each particle is…