Related papers: A Universal Dynamic Program and Refined Existence …
Motivated by the Internet-of-things and sensor networks for cyberphysical systems, the problem of dynamic sensor activation for the tracking of a time-varying process is examined. The tradeoff is between energy efficiency, which decreases…
We present an alternative view for the study of optimal control of partially observed Markov Decision Processes (POMDPs). We first revisit the traditional (and by now standard) separated-design method of reducing the problem to fully…
Calculating optimal policies is known to be computationally difficult for Markov decision processes (MDPs) with Borel state and action spaces. This paper studies finite-state approximations of discrete time Markov decision processes with…
We consider multiperiod stochastic control problems with non-parametric uncertainty on the underlying probabilistic model. We derive a new metric on the space of probability measures, called the adapted $(p, \infty)$--Wasserstein distance…
Markov Decision Processes (MDPs) are a formal framework for modeling and solving sequential decision-making problems. In finite-time horizons such problems are relevant for instance for optimal stopping or specific supply chain problems,…
Decentralized stochastic control problems with local information involve problems where multiple agents and subsystems which are coupled via dynamics and/or cost are present. Typically, however, the dynamics of such couplings is complex and…
Within the framework of viscosity solution, we study the relationship between the maximum principle (MP) in [9] and the dynamic programming principle (DPP) in [10] for a fully coupled forward-backward stochastic controlled system (FBSCS)…
This paper studies discrete-time average-cost infinite-horizon Markov decision processes (MDPs) with Borel state and action sets. It introduces new sufficient conditions for { the} validity of optimality inequalities and optimality…
In this paper, we first establish the dynamic programming principle for stochastic optimal control problems defined on compact Riemannian manifolds without boundary. Subsequently, we derive the associated Hamilton-Jacobi-Bellman (HJB)…
In this paper we study the stochastic control problem of partially observed (multi-dimensional) stochastic system driven by both Brownian motions and fractional Brownian motions. In the absence of the powerful tool of Girsanov…
In this paper, we develop a method to automatically generate a control policy for a dynamical system modeled as a Markov Decision Process (MDP). The control specification is given as a Linear Temporal Logic (LTL) formula over a set of…
Stochastic dynamical systems are fundamental in state estimation, system identification and control. System models are often provided in continuous time, while a major part of the applied theory is developed for discrete-time systems.…
Persistent monitoring using robot teams is of interest in fields such as security, environmental monitoring, and disaster recovery. Performing such monitoring in a fully on-line decentralised fashion has significant potential advantages for…
In modern robotics, effectively computing optimal control policies under dynamically varying environments poses substantial challenges to the off-the-shelf parametric policy gradient methods, such as the Deep Deterministic Policy Gradient…
Using a novel toy nautical navigation environment, we show that dynamic programming can be used when only incomplete information about a partially observed Markov decision process (POMDP) is known. By incorporating uncertainty into our…
The paper proposes a new stochastic intervention control model conducted in various commodity and stock markets. The essence of the phenomenon of intervention is described in accordance with current economic theory. A review of papers on…
We extend the proof of the dynamic programming principle (DPP) for standard stochastic optimal control problems driven by general L\'{e}vy noises. Under appropriate assumptions, it is shown that the DPP still holds when the state process…
This paper develops a dynamic programming (DP) approach for decentralized stochastic optimal control problems with delayed sharing information patterns, which exhibits the fundamental Properties of classical DP of centralized partially…
In deterministic and stochastic control theory, relaxed or randomized control policies allow for versatile mathematical analysis (on continuity, compactness, convexity and approximations) to be applicable with no artificial restrictions on…
This paper presents a general class of dynamic stochastic optimization problems we refer to as Stochastic Depletion Problems. A number of challenging dynamic optimization problems of practical interest are stochastic depletion problems.…