Related papers: Linear response and moderate deviations: hierarchi…
Markov Decision Processes (MDPs) are a popular class of models suitable for solving control decision problems in probabilistic reactive systems. We consider parametric MDPs (pMDPs) that include parameters in some of the transition…
We prove large deviation principles (LDPs) for random matrices in the orthogonal group and Stiefel manifold, determining both the speed and good convex rate functions that are explicitly given in terms of certain log-determinants of…
For ${1/2}<\alpha<1$, we propose the MDP analysis for family $$ S^\alpha_n=\frac{1}{n^\alpha}\sum_{i=1}^nH(X_{i-1}), n\ge 1, $$ where $(X_n)_{n\ge 0}$ be a homogeneous ergodic Markov chain, $X_n\in \mathbb{R}^d$, when the spectrum of…
The random field Curie-Weiss model is derived from the classical Curie-Weiss model by replacing the deterministic global magnetic field by random local magnetic fields. This opens up a new and interestingly rich phase structure. In this…
In contrast to the study of Langevin equations in a homogeneous environment in the literature, the study on Langevin equations in randomly-varying environments is relatively scarce. Almost all the existing works require random environments…
For ${1/2}<\alpha<1$, we propose the MDP analysis for family $$ S^\alpha_n=\frac{1}{n^\alpha}\sum_{i=1}^nH(X_{i-1}), n\ge 1, $$ where $(X_n)_{n\ge 0}$ be a homogeneous ergodic Markov chain, $X_n\in \mathbb{R}^d$, when the spectrum of…
The main aim of this paper is to study the moderate deviation principle for McKean-Vlasov stochastic differential equations with multiple scales. Specifically, we are interested in the asymptotic estimates of the deviation processes…
The Hierarchical Dirichlet process is a discrete random measure serving as an important prior in Bayesian non-parametrics. It is motivated with the study of groups of clustered data. Each group is modelled through a level two Dirichlet…
This note provides a tool to infer moderate deviations principles for specific random variables from deviations principles for their Hubbard-Stratonovich transforms.
Robust Markov Decision Processes (MDPs) are a powerful framework for modeling sequential decision-making problems with model uncertainty. This paper proposes the first first-order framework for solving robust MDPs. Our algorithm interleaves…
Markov decision processes (MDPs) are known to be sensitive to parameter specification. Distributionally robust MDPs alleviate this issue by allowing for \emph{ambiguity sets} which give a set of possible distributions over parameter sets.…
We introduce Multi-Environment Markov Decision Processes (MEMDPs) which are MDPs with a set of probabilistic transition functions. The goal in a MEMDP is to synthesize a single controller with guaranteed performances against all…
A constrained Markov decision process (CMDP) approach is developed for response-adaptive procedures in clinical trials with binary outcomes. The resulting CMDP class of Bayesian response -- adaptive procedures can be used to target a…
It is common to address the curse of dimensionality in Markov decision processes (MDPs) by exploiting low-rank representations. This motivates much of the recent theoretical study on linear MDPs. However, most approaches require a given…
In this paper, we proved moderate deviation principles for a fully coupled two-time-scale stochastic systems, where the slow process is given by stochastic differential equations with small noise, while the fast process is a rapidly…
Markov decision processes (MDPs) are a popular model for decision-making in the presence of uncertainty. The conventional view of MDPs in verification treats them as state transformers with probabilities defined over sequences of states and…
We apply Lindeberg's method, invented to prove a central limit theorem, to analyze the moderate deviations around such a central limit theorem. In particular, we will show moderate deviation principles for martingales as well as for random…
The density-dependent Markov chain (DDMC) introduced in \cite{Kurtz1978} is a continuous time Markov process applied in fields such as epidemics, chemical reactions and so on. In this paper, we give moderate deviation principles of paths of…
We provide non-asymptotic, relative deviation bounds for the eigenvalues of empirical covariance and Gram matrices in general settings. Unlike typical uniform bounds, which may fail to capture the behavior of smaller eigenvalues, our…
Partially observable Markov decision processes (POMDPs) provide an elegant mathematical framework for modeling complex decision and planning problems in stochastic domains in which states of the system are observable only indirectly, via a…