Related papers: AoI-MDP: An AoI Optimized Markov Decision Process …
Data freshness, measured by Age of Information (AoI), is highly relevant in networked applications such as Vehicle to Everything (V2X), smart health systems, and Industrial Internet of Things (IIoT). However, freshness alone does not always…
Constrained Markov Decision Process (CMDP) is a natural framework for reinforcement learning tasks with safety constraints, where agents learn a policy that maximizes the long-term reward while satisfying the constraints on the long-term…
The recent development in Internet of Things necessitates caching of dynamic contents, where new versions of contents become available around-the-clock and thus timely update is required to ensure their relevance. The age of information…
Markov decision processes (MDPs) are a standard model for sequential decision-making problems and are widely used across many scientific areas, including formal methods and artificial intelligence (AI). MDPs do, however, come with the…
Age of Information (AoI) has attracted much attention recently due to its capability of characterizing the freshness of information. To improve information freshness over fading channels, efficient scheduling methods are highly desired for…
In the Internet of Vehicles (IoV), Age of Information (AoI) has become a vital performance metric for evaluating the freshness of information in communication systems. Although many studies aim to minimize the average AoI of the system…
Markov decision processes (MDPs) are the defacto frame-work for sequential decision making in the presence ofstochastic uncertainty. A classical optimization criterion forMDPs is to maximize the expected discounted-sum pay-off, which…
Most applications in autonomous navigation using mounted cameras rely on the construction and processing of geometric 3D point clouds, which is an expensive process. However, there is another simpler way to make a space navigable quickly:…
Age of Information (AoI) is a crucial metric for quantifying information freshness in real-time systems where the sampling rate of data packets is time-varying. Evaluating AoI under such conditions is challenging, as system states become…
The Internet of underwater things (IoUT) is envisioned to be an essential part of maritime activities. Given the IoUT devices' wide-area distribution and constrained transmit power, autonomous underwater vehicles (AUVs) have been widely…
We consider a discrete-time multi-channel network where the destination collects time-sensitive packets from multiple sources with sided channel information. The popular metric, Age of Information (AoI), is applied to measure the data…
We introduce the active exploration problem in Markov decision processes (MDPs). Each state of the MDP is characterized by a random value and the learner should gather samples to estimate the mean value of each state as accurately as…
Using multiple sensors to update the status process of interest is promising in improving the information freshness. The unordered arrival of status updates at the monitor end poses a significant challenge in analyzing the timeliness…
Markov decision processes (MDPs) are widely used for modeling decision-making problems in robotics, automated control, and economics. Traditional MDPs assume that the decision maker (DM) knows all states and actions. However, this may not…
Markov decision processes (MDPs) are widely used for modeling decision-making problems in robotics, automated control, and economics. Traditional MDPs assume that the decision maker (DM) knows all states and actions. However, this may not…
The Age-of-Information is an important metric for investigating the timeliness performance in information-update systems. In this paper, we study the AoI minimization problem under a new Pull model with replication schemes, where a user…
Model-based learning algorithms have been shown to use experience efficiently when learning to solve Markov Decision Processes (MDPs) with finite state and action spaces. However, their high computational cost due to repeatedly solving an…
As Internet of Things (IoT) systems scale and device heterogeneity grows, multimodal data have become ubiquitous. Meanwhile, evaluating the freshness of multimodal data is essential, as stale updates would delay task execution, degrade…
In this paper, we consider the cellular Internet of unmanned aerial vehicles (UAVs), where UAVs sense data for multiple tasks and transmit the data to the base station (BS). To quantify the "freshness" of the data at the BS, we bring in the…
Measuring states in reinforcement learning (RL) can be costly in real-world settings and may negatively influence future outcomes. We introduce the Actively Observable Markov Decision Process (AOMDP), where an agent not only selects control…