Related papers: On the Optimality Equation for Average Cost Markov…
We consider a stochastic inventory control problem under censored demands, lost sales, and positive lead times. This is a fundamental problem in inventory management, with significant literature establishing near-optimality of a simple…
A purely state-dependent cost function can be modified by introducing a control-dependent term rewarding submaximal control utilization. A moderation incentive is identically zero on the boundary of the admissible control region and…
Learning and optimal control under robust Markov decision processes (MDPs) have received increasing attention, yet most existing theory, algorithms, and applications focus on finite-horizon or discounted models. Long-run average-reward…
We consider the problem of planning the aggregate energy consumption for a set of thermostatically controlled loads for demand response, accounting price forecast trajectory and thermal comfort constraints. We address this as a…
In this paper, we consider a modified version of the control problem in a model free Markov decision process (MDP) setting with large state and action spaces. The control problem most commonly addressed in the contemporary literature is to…
We propose a formulation of the stochastic cutting stock problem as a discounted infinite-horizon Markov decision process. At each decision epoch, given current inventory of items, an agent chooses in which patterns to cut objects in stock…
Calculating optimal policies is known to be computationally difficult for Markov decision processes (MDPs) with Borel state and action spaces. This paper studies finite-state approximations of discrete time Markov decision processes with…
Many control problems in environments that can be modeled as Markov decision processes (MDPs) concern infinite-time horizon specifications. The classical aim in this context is to compute a control policy that maximizes the probability of…
We consider the problem of computing optimal policies in average-reward Markov decision processes. This classical problem can be formulated as a linear program directly amenable to saddle-point optimization methods, albeit with a number of…
The paper [12] examines a concept of equilibrium policies instead of optimal controls in stochastic optimization to analyze a mean-variance portfolio selection problem. We follow the same approach in order to investigate the Merton…
The paper considers the optimal control problem of inventory of a discrete product in regeneration scheme with a Poisson flow of customer requirements. In the system deferred demand is allowed, the volume of which is limited by a given…
We consider a risk-sensitive continuous-time Markov decision process over a finite time duration. Under the conditions that can be satisfied by unbounded transition and cost rates, we show the existence of an optimal policy, and the…
We study the error introduced by entropy regularization in infinite-horizon discrete discounted Markov decision processes. We show that this error decreases exponentially in the inverse regularization strength, both in a weighted…
In this paper we consider a control problem for a Partially Observable Piecewise Deterministic Markov Process of the following type: After the jump of the process the controller receives a noisy signal about the state and the aim is to…
This article treats both discrete time and continuous time stopping problems for general Markov processes on the real line with general linear costs. Using an auxiliary function of maximum representation type, conditions are given to…
The problem of constrained Markov decision process is considered. An agent aims to maximize the expected accumulated discounted reward subject to multiple constraints on its costs (the number of constraints is relatively small). A new dual…
We study the continuity properties of optimal solutions to stochastic control problems with respect to initial probability measures and applications of these to the robustness of optimal control policies applied to systems with incomplete…
In this paper, we study the offline sequential feature-based pricing and inventory control problem where the current demand depends on the past demand levels and any demand exceeding the available inventory is lost. Our goal is to leverage…
Long-run average optimization problems for Markov decision processes (MDPs) require constructing policies with optimal steady-state behavior, i.e., optimal limit frequency of visits to the states. However, such policies may suffer from…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…