Related papers: Learning to Stop with Surprisingly Few Samples
We solve the problem of optimal stopping of a Brownian motion subject to the constraint that the stopping time's distribution is a given measure consisting of finitely-many atoms. In particular, we show that this problem can be converted to…
We study the problem of determining an effective exploration strategy in static and non-linear optimization problems, which depend on an unknown scalar parameter to be learned from online collected noisy data. An optimal trade-off between…
In stochastic systems, numerically sampling the relevant trajectories for the estimation of the large deviation statistics of time-extensive observables requires overcoming their exponential (in space and time) scarcity. The optimal way to…
This note re-visits the rolling-horizon control approach to the problem of a Markov decision process (MDP) with infinite-horizon discounted expected reward criterion. Distinguished from the classical value-iteration approach, we develop an…
We present a new algorithm based on posterior sampling for learning in constrained Markov decision processes (CMDP) in the infinite-horizon undiscounted setting. The algorithm achieves near-optimal regret bounds while being advantageous…
In Robbins' problem of minimizing the expected rank, a finite sequence of $n$ independent, identically distributed random variables are observed sequentially and the objective is to stop at such a time that the expected rank of the selected…
We consider the problem where an active Decision-Maker (DM) is tasked to identify the true hypothesis using as few samples as possible while maintaining accuracy. The DM collects samples according to its determined actions and knows the…
We prove new upper and lower bounds for sample complexity of finding an $\epsilon$-optimal policy of an infinite-horizon average-reward Markov decision process (MDP) given access to a generative model. When the mixing time of the…
This paper extends the core results of discrete time infinite horizon dynamic programming to the case of state-dependent discounting. We obtain a condition on the discount factor process under which all of the standard optimality results…
The infinite horizon setting is widely adopted for problems of reinforcement learning (RL). These invariably result in stationary policies that are optimal. In many situations, finite horizon control problems are of interest and for such…
Energy storage scheduling problems, where a storage is operated to maximize its profit in response to a price signal, are essentially infinite-horizon optimization problems as storage systems operate continuously, without a foreseen end to…
We study the numerical solution of nonlinear partially observed optimal stopping problems. The system state is taken to be a multi-dimensional diffusion and drives the drift of the observation process, which is another multi-dimensional…
Optimal stopping problems consider the question of deciding when to stop an observation-generating process in order to maximize a return. We examine the problem of simultaneously learning and planning in such domains, when data is collected…
We propose information-directed sampling -- a new approach to online optimization problems in which a decision-maker must balance between exploration and exploitation while learning from partial feedback. Each action is sampled in a manner…
This paper studies a finite horizon utility maximization problem on excessive consumption under a drawdown constraint. Our control problem is an extension of the one considered in Bahman et al. (2019) to the model with a finite horizon and…
This paper proposes a receding horizon active learning and control problem for dynamical systems in which Gaussian Processes (GPs) are utilized to model the system dynamics. The active learning objective in the optimization problem is…
We study reward-free and reward-agnostic exploration in episodic finite-horizon Markov decision processes (MDPs), where an agent explores an unknown environment without observing external rewards. Reward-free exploration aims to enable…
We present a new algorithm for imitation learning in infinite horizon linear MDPs dubbed ILARL which greatly improves the bound on the number of trajectories that the learner needs to sample from the environment. In particular, we remove…
We study infinite-horizon Discounted Markov Decision Processes (DMDPs) under a generative model. Motivated by the Algorithm with Advice framework Mitzenmacher and Vassilvitskii 2022, we propose a novel framework to investigate how a…
This paper proposes a computationally tractable algorithm for learning infinite-horizon average-reward linear mixture Markov decision processes (MDPs) under the Bellman optimality condition. Our algorithm for linear mixture MDPs achieves a…