Related papers: Asymptotics of impulse control problem with multip…
This article treats long term average impulse control problems with running costs in the case that the underlying process is a L\'evy process. Under quite general conditions we characterize the value of the control problem as the value of a…
This paper analyzes and explicitly solves a class of long-term average impulse control problems and a related class of singular control problems. The underlying process is a general one-dimensional diffusion with appropriate boundary…
Bellman equations of ergodic type related to risk-sensitive control are considered. We treat the case that the nonlinear term is positive quadratic form on first-order partial derivatives of solution, which includes linear exponential…
In this paper, which is a continuation of the previously published discrete time paper we develop a theory for continuous time stochastic control problems which, in various ways, are time inconsistent in the sense that they do not admit a…
Markov decision processes are typically used for sequential decision making under uncertainty. For many aspects however, ranging from constrained or safe specifications to various kinds of temporal (non-Markovian) dependencies in task and…
A multiplicative relative value iteration algorithm for solving the dynamic programming equation for the risk-sensitive control problem is studied for discrete time controlled Markov chains with a compact Polish state space, and controlled…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
We consider the problem of maximizing the expected average reward obtained over an infinite time horizon by $n$ weakly coupled Markov decision processes. Our setup is a substantial generalization of the multi-armed restless bandit problem…
This work contributes to the field of optimal control of bilinear systems. It concerns a continuous time, finite dimensional, bilinear state equation with a quadratic performance index to be minimized. The state equation is non-autonomous…
A class of stochastic optimal control problems involving optimal stopping is considered. Methods of Krylov are adapted to investigate the numerical solutions of the corresponding normalized Bellman equations and to estimate the rate of…
The main objective of this paper is the construction of the solution of an impulsive stochastic differential equation, subject to control conditions in the pulse-times and give sufficient conditions for them to be random variables with…
We consider an optimal control problem for piecewise deterministic Markov processes (PDMPs) on a bounded state space. The control problem under study is very general: a pair of controls acts continuously on the deterministic flow and on the…
We introduce a notion of bounded variation solution for a new class of nonlinear control systems with ordinary and impulsive controls, in which the drift function depends not only on the state, but also on its past history, through a finite…
We present a new, tractable method for solving and analyzing risk-aware control problems over finite and infinite, discounted time-horizons where the dynamics of the controlled process are described as a martingale problem. Supposing…
We propose using an adaptive sampling method to detect changes for a system with multiple lines. The adaptive sampling utilizes the information in responses to learn on which line is more likely to have a change thus allocating more units…
While there is an extensive body of research on the analysis of Value Iteration (VI) for discounted cumulative-reward MDPs, prior work on analyzing VI for (undiscounted) average-reward MDPs has been limited, and most prior results focus on…
We consider a classical finite horizon optimal control problem for continuous-time pure jump Markov processes described by means of a rate transition measure depending on a control parameter and controlled by a feedback law. For this class…
We consider a nonlinear control system with vector-valued measures as controls and with dynamics depending on time delayed states. First, we introduce a notion of discontinuous, bounded variation solution associated with this system and…
In this paper we consider an infinite time horizon risk-sensitive optimal stopping problem for a Feller--Markov process with an unbounded terminal cost function. We show that in the unbounded case an associated Bellman equation may have…
We consider the non-linear optimal multiple stopping problem under general conditions on the non-linear evaluation operators, which might depend on two time indices: the time of evaluation/assessment and the horizon (when the reward or loss…