Related papers: A pseudo-Markov property for controlled diffusion …
In this work, we investigate the optimal control problem for continuous-time Markov decision processes with the random impact of the environment. We provide conditions to show the existence of optimal controls under finite-horizon criteria.…
We present a new, tractable method for solving and analyzing risk-aware control problems over finite and infinite, discounted time-horizons where the dynamics of the controlled process are described as a martingale problem. Supposing…
In this paper, we establish a general stochastic maximum principle for optimal control for systems described by a continuous-time Markov regime-switching stochastic recursive utilities model. The control domain is postulated not to be…
We show that a one-dimensional regular continuous Markov process \(\X\) with scale function \(s\) is a Feller--Dynkin process precisely if the space transformed process \(s (X)\) is a martingale when stopped at the boundaries of its state…
We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…
We consider the task of filtering a dynamic parameter evolving as a diffusion process, given data collected at discrete times from a likelihood which is conjugate to the marginal law of the diffusion, when a generic dual process on a…
For controlled discrete-time stochastic processes we introduce a new class of dynamic risk measures, which we call process-based. Their main features are that they measure risk of processes that are functions of the history of a base…
This paper focuses on optimizing probabilities of events of interest defined over general controlled discrete-time Markov processes. It is shown that the optimization over a wide class of $\omega$-regular properties can be reduced to the…
We consider a class of optimal control problems, with finite or infinite horizon, for a continuous-time Markov chain with finite state space. In this case, the control process affects the transition rates. We suppose that the controlled…
In this brief note, we investigate some constructions of Lyapunov functions for stochastic discrete-time stabilizable dynamical systems, in other words, controlled Markov chains. The main question here is whether a Lyapunov function in some…
For a wide class of continuous-time Markov processes, including all irreducible hypoelliptic diffusions evolving on an open, connected subset of $\RL^d$, the following are shown to be equivalent: (i) The process satisfies (a slightly weaker…
Information relaxation and duality in Markov decision processes have been studied recently by several researchers with the goal to derive dual bounds on the value function. In this paper we extend this dual formulation to controlled Markov…
Stochastic convergence of discrete time Markov processes has been analysed based on a dual Lyapunov approach. Using some existing results on ergodic theory of Markov processes, it has been shown that existence of a properly subinvariant…
We consider a class of stochastic impulse control problems of general stochastic processes i.e. not necessarily Markovian. Under fairly general conditions we establish existence of an optimal impulse control. We also prove existence of…
We provide verification theorems (at different levels of generality) for infinite horizon stochastic control problems in continuous time for semimartingales. The control framework is given as an abstract "martingale formulation", which…
We consider a hidden Markov model, where the signal process, given by a diffusion, is only indirectly observed through some noisy measurements. The article develops a variational method for approximating the hidden states of the signal…
This paper presents a systematic approach to the advanced control of continuous crystallization processes using model predictive control. We provide a tutorial introduction to controlling complex particle size distributions by integrating…
Leveraging recent work on data-driven methods for constructing a finite state space Markov process from dynamical systems, we address two problems for obtaining further reduced statistical representations. The first problem is to extract…
We prove the dynamic programming principle for a class of diffusion processes controlled up to the time of exit from a cylindrical region $[0,T)\times G$. It is assumed that the functional to be maximized is in the Lagrange form with…
Bertrand et al. introduced a model of parameterised systems, where each agent is represented by a finite state system, and studied the following control problem: for any number of agents, does there exist a controller able to bring all…