Related papers: MDP-based Energy-aware Task Scheduling for Battery…

In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Building upon recent advances in Markov chain approximation methods and sampling-based algorithms for deterministic path planning,…

In this work, communication over a time-correlated point-to-point wireless channel is studied for an energy harvesting (EH) transmitter. In this model, we take into account the time and energy cost of acquiring channel state information. At…

We address the problem of real-time remote tracking of a partially observable Markov source in an energy harvesting system with an unreliable communication channel. We consider both sampling and transmission costs. Different from most prior…

Joint optimization of scheduling and estimation policies is considered for a system with two sensors and two non-collocated estimators. Each sensor produces an independent and identically distributed sequence of random variables, and each…

In this paper, we develop optimal policies for deciding when a wireless node with radio frequency (RF) energy harvesting (EH) capabilities should try and harvest ambient RF energy. While the idea of RF-EH is appealing, it is not always…

Internet of Things (IoT) devices often come with batteries of limited capacity that are not easily replaceable or rechargeable, and that constrain significantly the sensing, computing, and communication tasks that they can perform. The…

We propose a coordinated random access scheme for industrial internet-of-things (IIoT) scenarios, with machine-type devices (MTDs) generating sporadic correlated traffic. This occurs, e.g., when external events trigger data generation at…

In many practical sequential decision-making problems, tracking the state of the environment incurs a sensing/communication/computation cost. In these settings, the agent's interaction with its environment includes the additional component…

Following the occurrence of an extreme natural or man-made event, community recovery management should aim at providing optimal restoration policies for a community over a planning horizon. Calculating such optimal restoration polices in…

We consider the problem of optimally utilizing $N$ resources, each in an unknown binary state. The state of each resource can be inferred from state-dependent noisy measurements. Depending on its state, utilizing a resource results in…

This paper investigates the optimization problem of an infinite stage discrete time Markov decision process (MDP) with a long-run average metric considering both mean and variance of rewards together. Such performance metric is important…

Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…

In many practical applications, decision-making processes must balance the costs of acquiring information with the benefits it provides. Traditional control systems often assume full observability, an unrealistic assumption when…

We present a framework to address a class of sequential decision making problems. Our framework features learning the optimal control policy with robustness to noisy data, determining the unknown state and action parameters, and performing…

Massive internet of things microservices require integrating renewable energy harvesting into mobile edge computing (MEC) for sustainable eScience infrastructures. Spatiotemporal mismatches between stochastic task arrivals and intermittent…

This paper investigates a discrete energy state transition model for energy harvesting (EH) in cell-free massive multiple-input-multiple-output (CF-mMIMO) networks. A Markov chain-based stochastic process is conceived to characterize the…

In this paper, we examine the internet of things system which is dedicated for smart cities, smart factory, and connected cars, etc. To support such systems in wide area with low power consumption, energy harvesting technology without wired…

This note re-visits the rolling-horizon control approach to the problem of a Markov decision process (MDP) with infinite-horizon discounted expected reward criterion. Distinguished from the classical value-iteration approach, we develop an…

Monotonic Partially Observable Markov Decision Processes (POMDPs), where the system state progressively decreases until a restorative action is performed, can be used to model sequential repair problems effectively. This paper considers the…

The Internet of Things paradigm envisages the presence of many battery-powered sensors and this entails the design of energy-aware protocols. Source coding techniques allow to save some energy by compressing the packets sent over the…