Related papers: Structure and Optimality of Myopic Policy in Oppor…
We consider a system where a human operator processes a sequence of tasks that are similar in nature under a total time constraint. In these systems, the performance of the operator depends on its past utilization. This is akin to…
We study the capacity of discrete memoryless many-to-one interference channels, i.e., K user interference channels where only one receiver faces interference. For a class of many-to-one interference channels, we identify a noisy…
We study a novel multi-armed bandit problem that models the challenge faced by a company wishing to explore new strategies to maximize revenue whilst simultaneously maintaining their revenue above a fixed baseline, uniformly over time.…
We propose minimum empirical divergence (MED) policy for the multiarmed bandit problem. We prove asymptotic optimality of the proposed policy for the case of finite support models. In our setting, Burnetas and Katehakis has already proposed…
Developing an efficient spectrum access policy enables cognitive radios to dramatically increase spectrum utilization while ensuring predetermined quality of service levels for primary users. In this paper, modeling, performance analysis,…
When data are collected adaptively, such as in bandit algorithms, classical statistical approaches such as ordinary least squares and $M$-estimation will often fail to achieve asymptotic normality. Although recent lines of work have…
This paper considers designing an optimal policy for deadline-constrained access in cognitive radio networks, where a secondary user needs to complete a packet transmission over the vacant spectrum within a delivery deadline. To minimize…
We investigate the scheduling of a common resource between several concurrent users when the feasible transmission rate of each user varies randomly over time. Time is slotted and users arrive and depart upon service completion. This may…
Age-of-Information (AoI) is an application layer metric that has been widely adopted to quantify the information freshness of each information source. However, few works address the impact of probabilistic transmission failures on…
The fundamental problem of multiple secondary users contending for opportunistic spectrum access over multiple channels in cognitive radio networks has been formulated recently as a decentralized multi-armed bandit (D-MAB) problem. In a…
In ergodic singular stochastic control problems, a decision-maker can instantaneously adjust the evolution of a state variable using a control of bounded variation, with the goal of minimizing a long-term average cost functional. The cost…
Consider a multi-phase project management problem where the decision maker needs to deal with two issues: (a) how to allocate resources to projects within each phase, and (b) when to enter the next phase, so that the total expected reward…
This paper presents a general class of dynamic stochastic optimization problems we refer to as Stochastic Depletion Problems. A number of challenging dynamic optimization problems of practical interest are stochastic depletion problems.…
This paper considers a sequential sensor scheduling and remote estimation problem with multiple communication channels. Departing from the classical remote estimation paradigm, which involves one communication channel (noiseless or noisy),…
The Greedy algorithm is the simplest heuristic in sequential decision problem that carelessly takes the locally optimal choice at each round, disregarding any advantages of exploring and/or information gathering. Theoretically, it is known…
This paper examines the average age minimization problem where only a fraction of the network users can transmit simultaneously over unreliable channels. Finding the optimal scheduling scheme, in this case, is known to be challenging.…
We consider offline policy optimization (OPO) in contextual bandits, where one is given a fixed dataset of logged interactions. While pessimistic regularizers are typically used to mitigate distribution shift, prior implementations thereof…
In POMDPs, information about the hidden state, delivered through observations, is both valuable to the agent, allowing it to base its actions on better informed internal states, and a "curse", exploding the size and diversity of the…
The stochastic multi-armed bandit (MAB) problem is a common model for sequential decision problems. In the standard setup, a decision maker has to choose at every instant between several competing arms, each of them provides a scalar random…
We propose and study Collpasing Bandits, a new restless multi-armed bandit (RMAB) setting in which each arm follows a binary-state Markovian process with a special structure: when an arm is played, the state is fully observed, thus…