Related papers: Moderate deviation principle for ergodic Markov ch…

We resolve the open question regarding the sample complexity of policy learning for maximizing the long-run average reward associated with a uniformly ergodic Markov decision process (MDP), assuming a generative model. In this context, the…

Robust Markov Decision Processes (MDPs) are a powerful framework for modeling sequential decision-making problems with model uncertainty. This paper proposes the first first-order framework for solving robust MDPs. Our algorithm interleaves…

We study observation-based strategies for partially-observable Markov decision processes (POMDPs) with omega-regular objectives. An observation-based strategy relies on partial information about the history of a play, namely, on the past…

We consider the irreducibility of switch-based Markov chains for the approximate uniform sampling of Hamiltonian cycles in a given undirected dense graph on $n$ vertices. As our main result, we show that every pair of Hamiltonian cycles in…

We study the Markov chain Monte Carlo (MCMC) estimator for numerical integration for functions that do not need to be square integrable w.r.t. the invariant distribution. For chains with a spectral gap we show that the absolute mean error…

We consider the problem of learning the optimal policy for infinite-horizon Markov decision processes (MDPs). For this purpose, some variant of Stochastic Mirror Descent is proposed for convex programming problems with Lipschitz-continuous…

We establish the moderate deviation principle for the solutions of a class of stochastic partial differential equations with non-Lipschitz continuous coefficients. As an application, we derive the moderate deviation principle for two…

In Markov Decision Processes (MDPs) with intermittent state information, decision-making becomes challenging due to periods of missing observations. Linear programming (LP) methods can play a crucial role in solving MDPs, in particular,…

Let $(\xi_n)_{n=0}^\infty$ be a nonhomogeneous Markov chain taking values from finite state-space of $\mathbf{X}=\{1,2,\ldots,b\}$. In this paper, we will study the generalized entropy ergodic theorem with almost-everywhere and…

We present an alternative view for the study of optimal control of partially observed Markov Decision Processes (POMDPs). We first revisit the traditional (and by now standard) separated-design method of reducing the problem to fully…

We consider the problem of statistical inference in a parametric finite Markov chain model and develop a robust estimator of the parameters defining the transition probabilities via minimization of a suitable (empirical) version of the…

Markov decision process over vector addition system with states (VASS MDP) is a finite state model combining non-deterministic and probabilistic behavior, augmented with non-negative integer counters that can be incremented or decremented…

Let $\{X_n\}$ be a stationary and ergodic time series taking values from a finite or countably infinite set ${\cal X}$. Assume that the distribution of the process is otherwise unknown. We propose a sequence of stopping times $\lambda_n$…

A formula for the transition density of a Markov process defined by an infinite-dimensional stochastic equation is given in terms of the Ornstein--Uhlenbeck bridge and a useful lower estimate on the density is provided. As a consequence,…

We study the ergodic theory of a one-parameter family of interval maps T_alpha arising from generalized continued fraction algorithms. First of all, we prove the dependence of the metric entropy of T_alpha to be Hoelder-continuous in the…

We study robust Markov decision processes (RMDPs) with general policy parameterization under s-rectangular and non-rectangular uncertainty sets. Prior work is largely limited to tabular policies, and hence either lacks sample complexity…

We establish a moderate deviations principle (MDP) for the log-determinant $\log | \det (M_n) |$ of a Wigner matrix $M_n$ matching four moments with either the GUE or GOE ensemble. Further we establish Cram\'er--type moderate deviations and…

Let $\sigma(u)$, $u\in \mathbb{R}$ be an ergodic stationary Markov chain, taking a finite number of values $a_1,...,a_m$, and $b(u)=g(\sigma(u))$, where $g$ is a bounded and measurable function. We consider the diffusion type process $$…

We study a class of infinite-horizon average-cost Markov Decision Processes (MDPs) whose reward and transition structures are nearly separable. For the totally separable baseline (that is, with no perturbation), we derive an explicit…

We establish a moderate deviation principle (MDP) for the number of eigenvalues of a Wigner matrix in an interval close to the edge of the spectrum. Moreover we prove a MDP for the $i$th largest eigenvalue close to the edge. The proof…