Related papers: Dynamic Programming with State-Dependent Discounti…

We consider optimal stopping problems with finite-time horizon and state-dependent discounting. The underlying process is a one-dimensional linear diffusion and the gain function is time-homogeneous and difference of two convex functions.…

We present a continuous-time contract whereby a top-level player can incentivize a hierarchy of players below him to act in his best interest despite only observing the output of his direct subordinate. This paper extends Sannikov's…

Standard formulations of prescribed worst-case disturbance energy-gain control policies for linear time-varying systems depend on all forward model data. In discrete time, this dependence arises through a backward Riccati recursion. This…

We study a finite-horizon dynamic wholesale-price contract between a manufacturer and a retailer, both of whom observe only sales, rather than the true demand. When the retailer stocks out, unmet demand is unobserved, so both parties update…

This paper studies a finite-horizon Markov decision problem with information-theoretic constraints, where the goal is to minimize directed information from the controlled source process to the control process, subject to stage-wise cost…

We prove a general existence result in stochastic optimal control in discrete time where controls take values in conditional metric spaces, and depend on the current state and the information of past decisions through the evolution of a…

Model predictive control (MPC) schemes are commonly designed with fixed, i.e., time-invariant, horizon length and cost functions. If no stabilizing terminal ingredients are used, stability can be guaranteed via a sufficiently long horizon.…

This article is concerned with stability and performance of controlled stochastic processes under receding horizon policies. We carry out a systematic study of methods to guarantee stability under receding horizon policies via appropriate…

In this article, we consider the deterministic impulsively controlled system with infinite horizon and several discounted objective functionals. The constructed optimal control problem with functional constraints is reformulated as a Markov…

We study the sequential decision making problem of maximizing the expected total reward while satisfying a constraint on the expected total utility. We employ the natural policy gradient method to solve the discounted infinite-horizon…

We propose a novel approach to modeling advertising dynamics for a firm operating over distributed market domain based on controlled partial differential equations of diffusion type. Using our model, we consider a general type of…

We study dynamic discrete choice models, where a commonly studied problem involves estimating parameters of agent reward functions (also known as "structural" parameters), using agent behavioral data. Maximum likelihood estimation for such…

For a general optimal control problem for dynamical systems with hybrid dynamics, we study the dependency of the optimal cost and of the value function on the initial conditions, parameters, and perturbations. We show that upper and lower…

We revisit the finite time analysis of policy gradient methods in the one of the simplest settings: finite state and action MDPs with a policy class consisting of all stochastic policies and with exact gradient evaluations. There has been…

We consider the linear programming approach for constrained and unconstrained Markov decision processes (MDPs) under the long-run average cost criterion, where the class of MDPs in our study have Borel state spaces and discrete countable…

Optimization algorithms have a rich and fundamental relationship with ordinary differential equations given by its continuous-time limit. When the cost function varies with time -- typically in response to a dynamically changing environment…

This paper studies a dynamic optimal reinsurance and dividend-payout problem for an insurance company in a finite time horizon. The goal of the company is to maximize the expected cumulative discounted dividend payouts until bankruptcy or…

This paper introduces a novel Differential Dynamic Programming (DDP) algorithm for solving discrete-time finite-horizon optimal control problems with inequality constraints. Two variants, namely Feasible- and Infeasible-IPDDP algorithms,…

We consider an economic agent (a household or an insurance company) modelling its surplus process by a deterministic process or by a Brownian motion with drift. The goal is to maximise the expected discounted spendings/dividend payments,…

In this paper, we investigate the robustness of stationary mean-field equilibria in the presence of model uncertainties, specifically focusing on infinite-horizon discounted cost functions. To achieve this, we initially establish…