Related papers: Numerical method for optimal stopping of piecewise…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
Congestion pricing has become an effective instrument for traffic demand management on road networks. This paper proposes an optimal control approach for congestion pricing for day-to-day timescale that incorporates demand uncertainty and…
Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…
We consider a model of interacting neurons where the membrane potentials of the neurons are described by a multidimensional piecewise deterministic Markov process (PDMP) with values in ${\mathbb R}^N, $ where $ N$ is the number of neurons…
We study the quickest detection problem of a sudden change in the arrival rate of a Poisson process from a known value to an unknown and unobservable value at an unknown and unobservable disorder time. Our objective is to design an alarm…
The objective of this work is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite-time horizon discounted cost. The continuous-time controlled…
This paper solves a recursive optimal stopping problem with Poisson stopping constraints using the penalized backward stochastic differential equation (PBSDE) with jumps. Stopping in this problem is only allowed at Poisson random…
Control theory plays a pivotal role in understanding and optimizing the behavior of complex dynamical systems across various scientific and engineering disciplines. Two key frameworks that have emerged for modeling and solving control…
In order to give quantitative estimates for approximating the ergodic limit, we investigate probabilistic limit behaviors of time-averaging estimators of numerical discretizations for a class of time-homogeneous Markov processes, by…
Using the tools of the Markov Decision Processes, we justify the dynamic programming approach to the optimal impulse control of deterministic dynamical systems. We prove the equivalence of the integral and differential forms of the…
In this paper we consider stopping problems for continuous-time Markov chains under a general risk-sensitive optimization criterion for problems with finite and infinite time horizon. More precisely our aim is to maximize the certainty…
In this paper, we study the numerical method for stochastic optimal control problems (SOCPs). By reducing the optimal control problem to the discrete case, we derive a discrete stochastic maximum principle (SMP). With the help of this SMP,…
We introduce a class of numerical schemes for optimal control problems based on a novel Markov chain approximation, which uses, in turn, a piecewise constant policy approximation, Euler-Maruyama time stepping, and a Gauss-Hermite…
We investigate the Poisson regression method for Markov and semi-Markov jump processes from a nonparametric angle, allowing the lengths of the time and duration intervals in the partition to vary with the number of observations. Imposing no…
We study the problem of synthesizing a policy that maximizes the entropy of a Markov decision process (MDP) subject to a temporal logic constraint. Such a policy minimizes the predictability of the paths it generates, or dually, maximizes…
The fixed-horizon constrained Markov Decision Process (C-MDP) is a well-known model for planning in stochastic environments under operating constraints. Chance-Constrained MDP (CC-MDP) is a variant that allows bounding the probability of…
We present limit theorems for a sequence of Piecewise Deterministic Markov Processes (PDMPs) taking values in a separable Hilbert space. This class of processes provides a rigorous framework for stochastic spatial models in which discrete…
Differential Dynamic Programming (DDP) is an efficient trajectory optimization algorithm relying on second-order approximations of a system's dynamics and cost function, and has recently been applied to optimize systems with time-invariant…
The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…
Variance reduction methods are often needed for the reliability assessment of complex industrial systems, we focus on one variance reduction method in a given context, that is the interacting particle system method (IPS) used on piecewise…