Related papers: A note on weak compactness of occupation measures …
We consider Markov Decision Processes (MDPs) in which every stationary policy induces the same graph structure for the underlying Markov chain and further, the graph has the following property: if we replace each recurrent class by a node,…
Doubly robust methods hold considerable promise for off-policy evaluation in Markov decision processes (MDPs) under sequential ignorability: They have been shown to converge as $1/\sqrt{T}$ with the horizon $T$, to be statistically…
This paper provides new sufficient conditions so that the optimal policy of a partially observed Markov decision process (POMDP) can be lower bounded by a myopic policy. The two new proposed conditions, namely, Lehmann precision and…
Incorporating time-varying elements into electromagnetic systems has shown to be a powerful approach to challenge well-established performance limits, for example bounds on absorption and impedance matching. So far, the majority of these…
This paper addresses the problem of optimal control of robotic sensing systems aimed at autonomous information gathering in scenarios such as environmental monitoring, search and rescue, and surveillance and reconnaissance. The information…
The notion of $\Delta$-weakly mixing set is introduced, which shares similar properties of weakly mixing sets. It is shown that if a dynamical system has positive topological entropy, then the collection of $\Delta$-weakly mixing sets is…
We propose and analyze a temporal concatenation heuristic for solving large-scale finite-horizon Markov decision processes (MDP), which divides the MDP into smaller sub-problems along the time horizon and generates an overall solution by…
The aim of this paper is to prove ergodic decomposition theorems for probability measures quasi-invariant under Borel actions of inductively compact groups (Theorem 1) as well as for sigma-finite invariant measures (Corollary 1). For…
Simple random coverage models, well studied in Euclidean space, can also be defined on a general compact metric space. By analogy with the geometric models, and with the discrete coupon collector's problem and with cover times for finite…
For time-inconsistent stochastic controls in discrete time and finite horizon, an open problem in Bj\"ork and Murgoci (Finance Stoch, 2014) is the existence of an equilibrium control. A nonrandomized Borel measurable Markov equilibrium…
We study continuity and discontinuity of the upper and lower (modified) box-counting, Hausdorff, packing, (modified) correlation measure-dimension mappings under the weak, setwise and TV topology on the space of Borel measures respectively…
In this note we study a natural measure on plane partitions giving rise to a certain discrete-time Muttalib-Borodin process (MBP): each time-slice is a discrete version of a Muttalib-Borodin ensemble (MBE). The process is determinantal with…
This paper provides conditions under which total-cost and average-cost Markov decision processes (MDPs) can be reduced to discounted ones. Results are given for transient total-cost MDPs with tran- sition rates whose values may be greater…
We consider synchronizing properties of Markov decision processes (MDP), viewed as generators of sequences of probability distributions over states. A probability distribution is p-synchronizing if the probability mass is at least p in some…
This paper studies the synthesis of a joint control and active perception policy for a stochastic system modeled as a partially observable Markov decision process (POMDP), subject to temporal logic specifications. The POMDP actions…
In this work, we study the problem of actively classifying the attributes of dynamical systems characterized as a finite set of Markov decision process (MDP) models. We are interested in finding strategies that actively interact with the…
In this paper, we consider the gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We prove, under very…
We introduce Multi-Environment Markov Decision Processes (MEMDPs) which are MDPs with a set of probabilistic transition functions. The goal in a MEMDP is to synthesize a single controller with guaranteed performances against all…
Assume that $T$ is a conservative ergodic measure preserving transformation of the infinite measure space $(X,\mathcal{A},\mu)$.We study the asymptotic behaviour of occupation times of certain subsets of infinite measure. Specifically, we…
Based on a weak convergence argument, we provide a necessary and sufficient condition that guarantees that a nonnegative local martingale is indeed a martingale. Typically, conditions of this sort are expressed in terms of integrability…