Related papers: Proper Policies in Infinite-State Stochastic Short…
We consider both discrete and continuous control problems constrained by a fixed budget of some resource, which may be renewed upon entering a preferred subset of the state space. In the discrete case, we consider both deterministic and…
We propose a formulation of the stochastic cutting stock problem as a discounted infinite-horizon Markov decision process. At each decision epoch, given current inventory of items, an agent chooses in which patterns to cut objects in stock…
Current methods for solving Stochastic Shortest Path Problems (SSPs) find states' costs-to-go by applying Bellman backups, where state-of-the-art methods employ heuristics to select states to back up and prune. A fundamental limitation of…
In this manuscript we consider optimal control problems of stochastic differential equations with delays in the state and in the control. First, we prove an equivalent Markovian reformulation on Hilbert spaces of the state equation. Then,…
This paper considers optimal control of dynamical systems which are represented by nonlinear stochastic differential equations. It is well-known that the optimal control policy for this problem can be obtained as a function of a value…
Policy iteration is a widely used technique to solve the Hamilton Jacobi Bellman (HJB) equation, which arises from nonlinear optimal feedback control theory. Its convergence analysis has attracted much attention in the unconstrained case.…
We study the sample complexity of learning an $\epsilon$-optimal policy in the Stochastic Shortest Path (SSP) problem. We first derive sample complexity bounds when the learner has access to a generative model. We show that there exists a…
An optimal control problem is considered for a stochastic differential equation containing a state-dependent regime switching, with a recursive cost functional. Due to the non-exponential discounting in the cost functional, the problem is…
This paper studies an optimal stochastic impulse control problem in a finite horizon with a decision lag, by which we mean that after an impulse is made, a fixed number units of time has to be elapsed before the next impulse is allowed to…
This paper studies an optimal control problem for continuous-time stochastic systems subject to reachability objectives specified in a subclass of metric interval temporal logic specifications, a temporal logic with real-time constraints.…
We study the sample complexity of learning an $\epsilon$-optimal policy in the Stochastic Shortest Path (SSP) problem. We first derive sample complexity bounds when the learner has access to a generative model. We show that there exists a…
Stochastic shortest path (SSP) is a well-known problem in planning and control, in which an agent has to reach a goal state in minimum total expected cost. In the learning formulation of the problem, the agent is unaware of the environment…
Optimal stopping is the problem of determining when to stop a stochastic system in order to maximize reward, which is of practical importance in domains such as finance, operations management and healthcare. Existing methods for…
This paper presents a method to approximately solve stochastic optimal control problems in which the cost function and the system dynamics are polynomial. For stochastic systems with polynomial dynamics, the moments of the state can be…
This work considers infinite-horizon optimal control of positive linear systems applied to the case of network routing problems. We demonstrate the equivalence between Stochastic Shortest Path (SSP) problems and optimal control of a certain…
Stochastic Shortest Path problems (SSPs) are traditionally solved by computing each state's cost-to-go by applying Bellman backups. A Bellman backup updates a state's cost-to-go by iterating through every applicable action, computing the…
It is strange but fruitful to think about the functions as random processes. Any function can be viewed as a martingale (in many different ways) with discrete time. But it can be useful to have continuous time too. Processes can emulate…
Under a Bayesian framework, we formulate the fully sequential sampling and selection decision in statistical ranking and selection as a stochastic control problem, and derive the associated Bellman equation. Using value function…
This article is a continuation of a previous work where we studied infinite horizon control problems for which the dynamic, running cost and control space may be different in two half-spaces of some euclidian space $\R^N$. In this article…
We consider a finite horizon stochastic optimal control problem for nearest-neighbor random walk $\{X_i\}$ on the set of integers. The cost function is the expectation of exponential of the path sum of a random stationary and ergodic…