Related papers: A note on weak compactness of occupation measures …

We consider the control of a Markov decision process (MDP) that undergoes an abrupt change in its transition kernel (mode). We formulate the problem of minimizing regret under control-switching based on mode change detection, compared to a…

We consider a piecewise deterministic Markov decision process, where the expected exponential utility of total (nonnegative) cost is to be minimized. The cost rate, transition rate and post-jump distributions are under control. The state…

Many driven systems alternate between bursts of activity and quiescence and can become trapped in an absorbing state, such as complete inactivity in reaction-diffusion processes or extinction in predator-prey dynamics. It is generally…

A short proof is given of a necessary and sufficient condition for the normalized occupation measure of a L\'evy process in a metrizable compact group to be asymptotically uniform with probability one.

This paper discusses the functional stability of closed-loop Markov Chains under optimal policies resulting from a discounted optimality criterion, forming Markov Decision Processes (MDPs). We investigate the stability of MDPs in the sense…

Many problems in sequential decision making and stochastic control often have natural multiscale structure: sub-tasks are assembled together to accomplish complex goals. Systematically inferring and leveraging hierarchical structure,…

We consider Markov decision processes (MDPs) with unknown disturbance distribution and address this problem using the robust Markov decision process (RMDP) approach. We construct the empirical distribution of the unknown disturbance…

Analysis of Markov Decision Processes (MDP) is often hindered by state space explosion. Abstraction is a well-established technique in model checking to mitigate this issue. This paper presents a novel lazy abstraction method for MDP…

In many practical settings control decisions must be made under partial/imperfect information about the evolution of a relevant state variable. Partially Observable Markov Decision Processes (POMDPs) is a relatively well-developed framework…

This study focuses on solving the numerical challenges of imposing absorbing boundary conditions for dynamic simulations in the material point method (MPM). To attenuate elastic waves leaving the computational domain, the current work…

This article is concerned with moderate deviation principles of a general class of mean eld type interacting particle models. We discuss functional moderate deviations of the occupation measures for both the strong -topology on the space of…

In this paper, we propose a compositional approach for the construction of finite abstractions (a.k.a. finite Markov decision processes (MDPs)) for networks of discrete-time stochastic control subsystems that are not necessarily…

In Markov Decision Processes (MDPs) with intermittent state information, decision-making becomes challenging due to periods of missing observations. Linear programming (LP) methods can play a crucial role in solving MDPs, in particular,…

In this paper, we explore a topological system $f:M\rightarrow M$ with average shadowing property. We extend Sigmund's results and show that every non-empty, compact and connected subset $V\subseteq\mathcal {M}_{inv}(f)$ coincides with…

We study the quasi-stationary behavior of multidimensional processes absorbed when one of the coordinates vanishes. Our results cover competitive or weakly cooperative Lotka-Volterra birth and death processes and Feller diffusions with…

Packing topological entropy is a dynamical analogy of the packing dimension, which can be viewed as a counterpart of Bowen topological entropy. In the present paper, we will give a systematically study to the packing topological entropy for…

We introduce the active exploration problem in Markov decision processes (MDPs). Each state of the MDP is characterized by a random value and the learner should gather samples to estimate the mean value of each state as accurately as…

We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…

Markov decision processes (MDPs) are formal models commonly used in sequential decision-making. MDPs capture the stochasticity that may arise, for instance, from imprecise actuators via probabilities in the transition function. However, in…

Our aim is to find sufficient conditions for weak convergence of stochastic integrals with respect to the state occupation measure of a Markov chain. First, we study properties of the state indicator function and the state occupation…