Related papers: Time-Inconsistent Problems for Controlled Markov C…
We present a new, tractable method for solving and analyzing risk-aware control problems over finite and infinite, discounted time-horizons where the dynamics of the controlled process are described as a martingale problem. Supposing…
We provide a general approach to reformulating any continuous-time stochastic Stackelberg differential game under closed-loop strategies as a single-level optimisation problem with target constraints. More precisely, we consider a…
Arguing about the equilibrium distribution of continuous-time Markov chains can be vital for showing properties about the underlying systems. For example in biological systems, bistability of a chemical reaction network can hint at its…
We present a data-driven model predictive control scheme for chance-constrained Markovian switching systems with unknown switching probabilities. Using samples of the underlying Markov chain, ambiguity sets of transition probabilities are…
We study an infinite-horizon discrete-time optimal stopping problem under non-exponential discounting. A new method, which we call the iterative approach, is developed to find subgame perfect Nash equilibria. When the discount function…
We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite…
For an infinite-horizon continuous-time optimal stopping problem under non-exponential discounting, we look for an optimal equilibrium, which generates larger values than any other equilibrium does on the entire state space. When the…
This paper deals with the general discounted impulse control problem of a piecewise deterministic Markov process. We investigate a new family of epsilon-optimal strategies. The construction of such strategies is explicit and only…
We study the Merton portfolio management problem within a complete market, non constant time discount rate and general utility framework. The non constant discount rate introduces time inconsistency which can be solved by introducing sub…
A classical problem for Markov chains is determining their stationary (or steady-state) distribution. This problem has an equally classical solution based on eigenvectors and linear equation systems. However, this approach does not scale to…
This paper presents a new condition for the existence of optimal stationary policies in average-cost continuous-time Markov decision processes with unbounded cost and transition rates, arising from controlled queueing systems. This…
We derive sufficient and necessary optimality conditions in terms of a stochastic maximum principle (SMP) for controls associated with cost functionals of mean-field type, under dynamics driven by a class of Markov chains of mean-field type…
We present two related anytime algorithms for control of nonlinear systems when the processing resources available are time-varying. The basic idea is to calculate tentative control input sequences for as many time steps into the future as…
This work is devoted to finding the closed-loop equilibria for a class of mean-field games (MFGs) with infinitely many symmetric players in a common switching environment when the cost functional is under general discount in time. There are…
The linear response of a dynamical system refers to changes to properties of the system when small external perturbations are applied. We consider the little-studied question of selecting an optimal perturbation so as to (i) maximise the…
In this paper, we consider linear quadratic optimal control with mean-field type for discrete-time stochastic systems with state and control dependent noise. An optimal control problem is studied for a linear mean-field stochastic…
We formalize and analyze the notions of stochastic monotonicity and realizable mono-tonicity for Markov Chains in continuous-time, taking values in a finite partially ordered set. Similarly to what happens in discrete-time, the two notions…
One of the most classical games for stochastic processes is the zero-sum Dynkin (stopping) game. We present a complete equilibrium solution to a general formulation of this game with an underlying one-dimensional diffusion. A key result is…
We consider an infinite horizon optimal control problem for a continuous-time Markov chain $X$ in a finite set $I$ with noise-free partial observation. The observation process is defined as $Y_t = h(X_t)$, $t \geq 0$, where $h$ is a given…
This paper studies an optimal control problem for a string of vehicles with safety requirements and finite-time specifications on the approach time to a target region. Our problem formulation is motivated by scenarios involving autonomous…