Related papers: Optimality of Myopic Sensing in Multi-Channel Oppo…
A broadcast strategy for multiple access communication over slowly fading channels is introduced, in which the channel state information is known to only the receiver. In this strategy, the transmitters split their information streams into…
Partially observable Markov decision processes (POMDPs) provide an elegant mathematical framework for modeling complex decision and planning problems in stochastic domains in which states of the system are observable only indirectly, via a…
In this paper, we present a framework for distributively optimizing the transmission strategies of secondary users in an ad hoc cognitive radio network. In particular, the proposed approach allows secondary users to set their transmit…
Distributed opportunistic scheduling is studied for wireless ad-hoc networks, where many links contend for one channel using random access. In such networks, distributed opportunistic scheduling (DOS) involves a process of joint channel…
This paper considers the problem of channel sensing in cognitive radios. The system model considered is a set of N parallel (dis-similar) channels, where each channel at any given time is either available or occupied by a legitimate user.…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
Multi-user multi-armed bandits have emerged as a good model for uncoordinated spectrum access problems. In this paper we consider the scenario where users cannot communicate with each other. In addition, the environment may appear…
Planning plays an important role in the broad class of decision theory. Planning has drawn much attention in recent work in the robotics and sequential decision making areas. Recently, Reinforcement Learning (RL), as an agent-environment…
We introduce the active exploration problem in Markov decision processes (MDPs). Each state of the MDP is characterized by a random value and the learner should gather samples to estimate the mean value of each state as accurately as…
The problem of distributed access of a set of N on-off channels by K<N users is considered. The channels are slotted and modeled as independent but not necessarily identical alternating renewal processes. Each user decides to either observe…
For the multiple-input multiple-output (MIMO) broadcast channel with imperfect channel state information (CSI), neither the capacity nor the optimal transmission technique have been fully discovered. In this paper, we derive achievable…
A stochastic multi-user multi-armed bandit framework is used to develop algorithms for uncoordinated spectrum access. In contrast to prior work, it is assumed that rewards can be non-zero even under collisions, thus allowing for the number…
This paper investigates an Internet of Things (IoT) system in which multiple devices are observing some object's physical parameters and then offloading their observations back to the BS in time with opportunistic channel access.…
Learning a near optimal policy in a partially observable system remains an elusive challenge in contemporary reinforcement learning. In this work, we consider episodic reinforcement learning in a reward-mixing Markov decision process (MDP).…
Content caching in wireless networks provides a substantial opportunity to trade off low cost memory storage with energy consumption, yet finding the optimal causal policy with low computational complexity remains a challenge. This paper…
In many practical settings control decisions must be made under partial/imperfect information about the evolution of a relevant state variable. Partially Observable Markov Decision Processes (POMDPs) is a relatively well-developed framework…
We study planning problems where autonomous agents operate inside environments that are subject to uncertainties and not fully observable. Partially observable Markov decision processes (POMDPs) are a natural formal model to capture such…
We investigate the problem of designing optimal stealthy poisoning attacks on the control channel of Markov decision processes (MDPs). This research is motivated by the recent interest of the research community for adversarial and poisoning…
Effective capacity, which provides the maximum constant arrival rate that a given service process can support while satisfying statistical delay constraints, is analyzed in a multiuser scenario. In particular, the effective capacity region…
There is much interest in using partially observable Markov decision processes (POMDPs) as a formal model for planning in stochastic domains. This paper is concerned with finding optimal policies for POMDPs. We propose several improvements…