Related papers: Process level moderate deviations for stabilizing …

Robust Markov decision processes (MDPs) aim to handle changing or partially known system dynamics. To solve them, one typically resorts to robust optimization methods. However, this significantly increases computational complexity and…

We investigate partially observed Markov decision processes (POMDPs) with cost functions regularized by entropy terms describing state, observation, and control uncertainty. Standard POMDP techniques are shown to offer bounded-error…

We establish new conditions for obtaining uniform bounds on the moments of discrete-time stochastic processes. Our results require a weak negative drift criterion along with a state-dependent restriction on the sizes of the one-step jumps…

This paper is devoted to the estimation of the common marginal density function of weakly dependent processes. The accuracy of estimation is measured using pointwise risks. We propose a datadriven procedure using kernel rules. The bandwidth…

The fundamental functional summary statistics used for studying spatial point patterns are developed for marked homogeneous and inhomogeneous point processes on the surface of a sphere. These are extended to point processes on the surface…

The paper presents high-order accurate, energy-, and entropy-stable discretizations constructed from summation-by-parts (SBP) operators. Notably, the discretizations assemble global SBP operators and use continuous solutions, unlike…

We provide a framework for empirical process theory of locally stationary processes using the functional dependence measure. Our results extend known results for stationary Markov chains and mixing sequences by another common possibility to…

Point processes model the distribution of random point sets in mathematical spaces, such as spatial and temporal domains, with applications in fields like seismology, neuroscience, and economics. Existing statistical and machine learning…

Piecewise deterministic Markov processes (PDMPs) are a class of stochastic processes with applications in several fields of applied mathematics spanning from mathematical modeling of physical phenomena to computational methods. A PDMP is…

The problem of estimating a parameter in the drift coefficient is addressed for $N$ discretely observed independent and identically distributed stochastic differential equations (SDEs). This is done considering additional constraints,…

In this paper we consider the filtering of a class of partially observed piecewise deterministic Markov processes (PDMPs). In particular, we assume that an ordinary differential equation (ODE) drives the deterministic element and can only…

This paper discusses the functional stability of closed-loop Markov Chains under optimal policies resulting from a discounted optimality criterion, forming Markov Decision Processes (MDPs). We investigate the stability of MDPs in the sense…

The notion of statistical depth has been extensively studied in multivariate and functional data over the past few decades. In contrast, the depth on temporal point process is still under-explored. The problem is challenging because a point…

This work studies a two-time-scale functional system given by two jump-diffusions under the scale separation by a small parameter $\varepsilon \rightarrow 0$. The coefficients of the equations that govern the dynamics of the system depend…

Noisy sensing, imperfect control, and environment changes are defining characteristics of many real-world robot tasks. The partially observable Markov decision process (POMDP) provides a principled mathematical framework for modeling and…

We consider a collection of fully coupled weakly interacting diffusion processes moving in a two-scale environment. We study the moderate deviations principle of the empirical distribution of the particles' positions in the combined limit…

We present metrics for measuring the similarity of states in a finite Markov decision process (MDP). The formulation of our metrics is based on the notion of bisimulation for MDPs, with an aim towards solving discounted infinite horizon…

In this paper, a sparse Markov decision process (MDP) with novel causal sparse Tsallis entropy regularization is proposed.The proposed policy regularization induces a sparse and multi-modal optimal policy distribution of a sparse MDP. The…

In this review/tutorial article, we present recent progress on optimal control of partially observed Markov Decision Processes (POMDPs). We first present regularity and continuity conditions for POMDPs and their belief-MDP reductions, where…

Markov Decision Processes (MDPs) are a popular class of models suitable for solving control decision problems in probabilistic reactive systems. We consider parametric MDPs (pMDPs) that include parameters in some of the transition…