Related papers: Dynamic Programming with State-Dependent Discounti…
We present an efficient algorithm for model-free episodic reinforcement learning on large (potentially continuous) state-action spaces. Our algorithm is based on a novel $Q$-learning policy with adaptive data-driven discretization. The…
In this article we consider risk-sensitive control of semi-Markov processes with a discrete state space. We consider general utility functions and discounted cost in the optimization criteria. We consider random finite horizon and infinite…
The paper deals with a risk averse dynamic programming problem with infinite horizon. First, the required assumptions are formulated to have the problem well defined. Then the Bellman equation is derived, which may be also seen as a…
In this paper we consider a family of optimal control problems for economic models whose state variables are driven by Delay Differential Equations (DDE's). We consider two main examples: an AK model with vintage capital and an advertising…
For an infinite-horizon continuous-time optimal stopping problem under non-exponential discounting, we look for an optimal equilibrium, which generates larger values than any other equilibrium does on the entire state space. When the…
Dynamic decisions are pivotal to economic policy making. We show how existing evidence from randomized control trials can be utilized to guide personalized decisions in challenging dynamic environments with budget and capacity constraints.…
We consider adaptive decision-making problems where an agent optimizes a cumulative performance objective by repeatedly choosing among a finite set of options. Compared to the classical prediction-with-expert-advice set-up, we consider…
We analyze the convergence rate of various momentum-based optimization algorithms from a dynamical systems point of view. Our analysis exploits fundamental topological properties, such as the continuous dependence of iterates on their…
This papers deals with the constrained discounted control of piecewise deterministic Markov process (PDMPs) in general Borel spaces. The control variable acts on the jump rate and transition measure, and the goal is to minimize the total…
These lecture notes are derived from a graduate-level course in dynamic optimization, offering an introduction to techniques and models extensively used in management science, economics, operations research, engineering, and computer…
We prove the dynamic programming principle (DPP) in a class of problems where an agent controls a $d$-dimensional diffusive dynamics via both classical and singular controls and, moreover, is able to terminate the optimisation at a time of…
In this paper, we study the delayed stochastic recursive optimal control problem with a non-Lipschitz generator, in which both the dynamics of the control system and the recursive cost functional depend on the past path segment of the state…
This paper investigates a class of optimal control problems associated with Markov processes with local state information. The decision-maker has only local access to a subset of a state vector information as often encountered in…
This paper is devoted to studying an infinite time horizon stochastic recursive control problem with jumps, where infinite time horizon stochastic differential equation and backward stochastic differential equation with jumps describe the…
This paper is devoted to studying constrained continuous-time Markov decision processes (MDPs) in the class of randomized policies depending on state histories. The transition rates may be unbounded, the reward and costs are admitted to be…
The valuation process that economic agents undergo for investments with uncertain payoff typically depends on their statistical views on possible future outcomes, their attitudes toward risk, and, of course, the payoff structure itself.…
We study the computational complexity of the infinite-horizon discounted-reward Markov Decision Problem (MDP) with a finite state space $|\mathcal{S}|$ and a finite action space $|\mathcal{A}|$. We show that any randomized algorithm needs a…
In this paper, we study an infinite horizon non-autonomous stochastic recursive differential game. To this end, we first establish well-posedness and stability results for BSDEs with a time-dependent discount factor and a possibly unbounded…
We consider a piecewise deterministic Markov decision process, where the expected exponential utility of total (nonnegative) cost is to be minimized. The cost rate, transition rate and post-jump distributions are under control. The state…
In this research we study a finite horizon optimal purchasing problem for items with a mean reverting price process. Under this model a fixed amount of identical items are bought under a given deadline, with the objective of minimizing the…