Related papers: Sampling for Data Freshness Optimization: Non-line…
Received samples of a stochastic process are processed by a server for delivery as updates to a monitor. Each sample belongs to a class that specifies a distribution for its processing time and a function that describes how the value of the…
We consider the problem of controlling a fully specified Markov decision process (MDP), also known as the planning problem, when the state space is very large and calculating the optimal policy is intractable. Instead, we pursue the more…
The optimal transmission strategy to minimize the weighted combination of age of information (AoI) and total energy consumption is studied in this paper. It is assumed that the status update information is obtained and transmitted at fixed…
This paper investigates a multiuser scheduling problem within an uplink multiple-input multi-output (MIMO) status update network, consisting of a multi-antenna base station (BS) and multiple single-antenna devices. The presence of multiple…
We study the $(\varepsilon, \delta)$-PAC policy identification problem in finite-horizon episodic Markov Decision Processes. Existing approaches provide finite-time guarantees for approximate settings ($\varepsilon>0$) but suffer from high…
In this paper, we consider a status updating system where the transmitter sends status updates of the signal it monitors to the destination through a rate-limited link. We consider the scenario where the status of the monitored signal only…
We introduce a novel approach based on stochastic optimization to find the optimal sampling distribution for the data-driven stability analysis of switched linear systems. Our goal is to address limitations of existing approaches, in…
The freshness of real-time status processing of time-sensitive information is crucial for several applications, including healthcare monitoring and autonomous vehicles. This freshness is considered in this paper for the system where…
This paper addresses the problem of learning optimal control policies for systems with uncertain dynamics and high-level control objectives specified as Linear Temporal Logic (LTL) formulas. Uncertainty is considered in the workspace…
In this work, a wireless broadcast network with a base station (BS) sending random time-sensitive information updates to multiple users with interference constraints is considered. The Age of Synchronization (AoS), namely the amount of time…
An informative measurement is the most efficient way to gain information about an unknown state. We present a first-principles derivation of a general-purpose dynamic programming algorithm that returns an optimal sequence of informative…
Many systems require frequent and regular updates of a certain information. These updates have to be transferred regularly from the source to the destination. We consider scenarios in which an old packet becomes completely obsolete, in the…
We revisit the identification of an $\varepsilon$-optimal policy in average-reward Markov Decision Processes (MDP). In such MDPs, two measures of complexity have appeared in the literature: the diameter, $D$, and the optimal bias span, $H$,…
As Internet of Things (IoT) systems scale and device heterogeneity grows, multimodal data have become ubiquitous. Meanwhile, evaluating the freshness of multimodal data is essential, as stale updates would delay task execution, degrade…
In this paper, we investigate the age minimization problem for a two-hop relay system, under a resource constraint on the average number of forwarding operations at the relay. We first design an optimal policy by modelling the considered…
We consider Markov decision processes (MDPs) with unknown disturbance distribution and address this problem using the robust Markov decision process (RMDP) approach. We construct the empirical distribution of the unknown disturbance…
Fueled by the rapid development of communication networks and sensors in portable devices, today many mobile users are invited by content providers to sense and send back real-time useful information (e.g., traffic observations and sensor…
In Internet of Things (IoT), the freshness of status updates is crucial for mission-critical applications. In this regard, it is suggested to quantify the freshness of updates by using Age of Information (AoI) from the receiver's…
This paper studies the problem of data collection for policy evaluation in Markov decision processes (MDPs). In policy evaluation, we are given a target policy and asked to estimate the expected cumulative reward it will obtain in an…
The demand for Internet services that require frequent updates through small messages, such as microblogging, has tremendously grown in the past few years. Although the use of such applications by domestic users is usually free, their…