Related papers: Utility Maximizing Sequential Sensing Over a Finit…
We study the optimization of the expected long-term reward in finite partially observable Markov decision processes over the set of stationary stochastic policies. In the case of deterministic observations, also known as state aggregation,…
We consider the problem of controlling a fully specified Markov decision process (MDP), also known as the planning problem, when the state space is very large and calculating the optimal policy is intractable. Instead, we pursue the more…
We study the offline data-driven sequential decision making problem in the framework of Markov decision process (MDP). In order to enhance the generalizability and adaptivity of the learned policy, we propose to evaluate each policy by a…
We study Markov decision processes (MDPs) with a countably infinite number of states. The $\limsup$ (resp. $\liminf$) threshold objective is to maximize the probability that the $\limsup$ (resp. $\liminf$) of the infinite sequence of…
Allocating scarce resources among agents to maximize global utility is, in general, computationally challenging. We focus on problems where resources enable agents to execute actions in stochastic environments, modeled as Markov decision…
We introduce a linear space of finitely additive measures to treat the problem of optimal expected utility from consumption under a stochastic clock and an unbounded random endowment process. In this way we establish existence and…
We study the problem of determining an effective exploration strategy in static and non-linear optimization problems, which depend on an unknown scalar parameter to be learned from online collected noisy data. An optimal trade-off between…
Partial monitoring is an expressive framework for sequential decision-making with an abundance of applications, including graph-structured and dueling bandits, dynamic pricing and transductive feedback models. We survey and extend recent…
In applications of offline reinforcement learning to observational data, such as in healthcare or education, a general concern is that observed actions might be affected by unobserved factors, inducing confounding and biasing estimates…
Decision-making under uncertainty is a crucial ability for autonomous systems. In its most general form, this problem can be formulated as a Partially Observable Markov Decision Process (POMDP). The solution policy of a POMDP can be…
This article provides an introductory tutorial on structural results in partially observed Markov decision processes (POMDPs). Typically, computing the optimal policy of a POMDP is computationally intractable. We use lattice program- ming…
We introduce a linear space of finitely additive measures to treat the problem of optimal expected utility from consumption under a stochastic clock and an unbounded random endowment process. In this way we establish existence and…
In this paper, we consider joint antenna selection and optimal beamforming for energy efficient delay minimization. We assume multiple-input multi-output (MIMO) system with full duplex simultaneous wireless information and power transfer…
The agent learns to organize decision behavior to achieve a behavioral goal, such as reward maximization, and reinforcement learning is often used for this optimization. Learning an optimal behavioral strategy is difficult under the…
In this paper, we investigate dynamic optimization problems featuring both stochastic control and optimal stopping in a finite time horizon. The paper aims to develop new methodologies, which are significantly different from those of mixed…
In this paper, we consider the gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We prove, under very…
In the optimization of dynamic systems, the variables typically have constraints. Such problems can be modeled as a Constrained Markov Decision Process (CMDP). This paper considers the peak Constrained Markov Decision Process (PCMDP), where…
In this paper, adaptive training beam sequence design for efficient channel estimation in large millimeter-wave(mmWave) multiple-input multiple-output (MIMO) channels is considered. By exploiting the sparsity in large mmWave MIMO channels…
Memoryless and finite-memory policies offer a practical alternative for solving partially observable Markov decision processes (POMDPs), as they operate directly in the output space rather than in the high-dimensional belief space. However,…
This paper formulates and solves a sequential detection problem that involves the mutual information (stochastic observability) of a Gaussian process observed in noise with missing measurements. The main result is that the optimal decision…