Related papers: On the Optimality Equation for Average Cost Markov…

In this paper, we present a generalization of the certainty equivalence principle of stochastic control. One interpretation of the classical certainty equivalence principle for linear systems with output feedback and quadratic costs is as…

Control systems involving unknown parameters appear a natural framework for applications in which the model design has to take into account various uncertainties. In these circumstances the performance criterion can be given in terms of an…

In this paper, we consider an infinite horizon, continuous-review, stochastic inventory system in which cumulative customers' demand is price-dependent and is modeled as a Brownian motion. Excess demand is backlogged. The revenue is earned…

We study continuity and robustness properties of infinite-horizon average expected cost problems with respect to (controlled) transition kernels, and applications of these results to the problem of robustness of control policies designed…

For an infinite-horizon continuous-time optimal stopping problem under non-exponential discounting, we look for an optimal equilibrium, which generates larger values than any other equilibrium does on the entire state space. When the…

We consider the single-item single-stocking location stochastic inventory system under a fixed ordering cost component. A long-standing problem is that of determining the structure of the optimal control policy when this system is subject…

The paper \cite{helm:17} studies an inventory management problem under a long-term average cost criterion using weak convergence methods applied to average expected occupation and average expected ordering measures. Under the natural…

This work studies discrete-time discounted Markov decision processes with continuous state and action spaces and addresses the inverse problem of inferring a cost function from observed optimal behavior. We first consider the case in which…

In this paper we consider impulse control of continuous time Markov processes with average cost per unit time functional. This problem is approximated using impulse control problems stopped at the first exit time from increasing sequence of…

This paper concerns discrete-time infinite-horizon stochastic control systems with Borel state and action spaces and universally measurable policies. We study optimization problems on strategic measures induced by the policies in these…

A general backward stochastic linear-quadratic optimal control problem is studied, in which both the state equation and the cost functional contain the nonhomogeneous terms. The main feature of the problem is that the weighting matrices in…

This paper studies a finite-fuel two-dimensional degenerate singular stochastic control problem under regime switching that is motivated by the optimal irreversible extraction problem of an exhaustible commodity. A company extracts a…

We consider an optimal stochastic impulse control problem over an infinite time horizon motivated by a model of irreversible investment choices with fixed adjustment costs. By employing techniques of viscosity solutions and relying on…

We obtain an exact necessary and sufficient condition for the existence and uniqueness of equilibrium asset prices in infinite horizon, discrete-time, arbitrage free environments. Through several applications we show how the condition…

This paper analyzes single-item continuous-review inventory models with random supplies in which the inventory dynamic between orders is described by a diffusion process, and a long-term average cost criterion is used to evaluate decisions.…

Policy gradient methods are widely used in reinforcement learning. Yet, the nonconvexity of policy optimization poses significant challenges in understanding the global convergence of policy gradient methods. For a class of finite-horizon…

This paper studies a large number of homogeneous Markov decision processes where the transition probabilities and costs are coupled in the empirical distribution of states (also called mean-field). The state of each process is not known to…

We consider the problem of controlling a Markov decision process (MDP) with a large state space, so as to minimize average cost. Since it is intractable to compete with the optimal policy for large scale problems, we pursue the more modest…

We study a class of infinite-horizon average-cost Markov Decision Processes (MDPs) whose reward and transition structures are nearly separable. For the totally separable baseline (that is, with no perturbation), we derive an explicit…

We study the synthesis of a policy in a Markov decision process (MDP) following which an agent reaches a target state in the MDP while minimizing its total discounted cost. The problem combines a reachability criterion with a discounted…