Related papers: Ranking and Selection as Stochastic Control
We model the joint distribution of choice probabilities and decision times in binary choice tasks as the solution to a problem of optimal sequential sampling, where the agent is uncertain of the utility of each action and pays a constant…
In this paper we study the optimal stochastic control problem for a path-dependent stochastic system under a recursive path-dependent cost functional, whose associated Bellman equation from dynamic programming principle is a path-dependent…
In this work, we investigate a stochastic control framework for global optimization over both Euclidean spaces and the Wasserstein space of probability measures, where the objective function may be non-convex and/or non-differentiable. In…
Adaptive optimal control of nonlinear dynamic systems with deterministic and known dynamics under a known undiscounted infinite-horizon cost function is investigated. Policy iteration scheme initiated using a stabilizing initial control is…
We consider a general class of dynamic resource allocation problems within a stochastic optimal control framework. This class of problems arises in a wide variety of applications, each of which intrinsically involves resources of different…
In a fixed budget ranking and Selection (R&S) problem, one aims to identify the best design among a finite number of candidates by efficiently allocating the given computing budget to evaluate design performance. Classical methods for R&S…
This survey is focused on certain sequential decision-making problems that involve optimizing over probability functions. We discuss the relevance of these problems for learning and control. The survey is organized around a framework that…
This paper studies stochastic optimization problems and associated Bellman equations in formats that allow for reduced dimensionality of the cost-to-go functions. In particular, we study stochastic control problems in the…
We present a unified framework for learning continuous control policies using backpropagation. It supports stochastic control by treating stochasticity in the Bellman equation as a deterministic function of exogenous noise. The product is a…
We consider the problem of jointly testing multiple hypotheses and estimating a random parameter of the underlying distribution. This problem is investigated in a sequential setup under mild assumptions on the underlying random process. The…
We study an optimal control problem in which both the objective function and the dynamic constraint contain an uncertain parameter. Since the distribution of this uncertain parameter is not exactly known, the objective function is taken as…
We describe an approximate dynamic programming approach to compute lower bounds on the optimal value function for a discrete time, continuous space, infinite horizon setting. The approach iteratively constructs a family of lower bounding…
We introduce AlphaRank, an artificial intelligence approach to address the fixed-budget ranking and selection (R&S) problems. We formulate the sequential sampling decision as a Markov decision process and propose a Monte Carlo…
Recently, it has been shown how sampling actions from the predictive distribution over the optimal action-sometimes called Thompson sampling-can be applied to solve sequential adaptive control problems, when the optimal policy is known for…
Dual control explicitly addresses the problem of trading off active exploration and exploitation in the optimal control of partially unknown systems. While the problem can be cast in the framework of stochastic dynamic programming, exact…
We present a neural network approach for approximating the value function of high-dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space…
Kernel embeddings of distributions have recently gained significant attention in the machine learning community as a data-driven technique for representing probability distributions. Broadly, these techniques enable efficient computation of…
We describe a new approach for managing aleatoric uncertainty in the Reinforcement Learning (RL) paradigm. Instead of selecting actions according to a single statistic, we propose a distributional method based on the second-order stochastic…
``When in a difficult situation, it is sometimes better to give up and start all over again''. While this empirical truth has been regularly observed in a wide range of circumstances, quantifying the effectiveness of such a heuristic…
We investigate an optimal control problem motivated by neuroscience, where the dynamics is driven by a Poisson process with a controlled stochastic intensity and an unknown parameter. Given a prior distribution for the unknown parameter, we…