Related papers: Markov Decision Processing Networks
This paper considers a multi-process and multi-controller wireless networked control system (WNCS). There are $N$ independent linear time-invariant processes in the system plant which represent different kinds of physical processes. By…
Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are…
Discrete-time Markov Chains (MCs) and Markov Decision Processes (MDPs) are two standard formalisms in system analysis. Their main associated quantitative objectives are hitting probabilities, discounted sum, and mean payoff. Although there…
Power grid load scheduling is a critical task that ensures the balance between electricity generation and consumption while minimizing operational costs and maintaining grid stability. Traditional optimization methods often struggle with…
We establish a connection between policy evaluation in Markov decision processes and PageRank in network analysis. For a fixed policy, we show that the value function of a discounted Markov decision process can be obtained, up to an…
We propose two scheduling algorithms that seek to optimize the quality of scalably coded videos that have been stored at a video server before transmission.} The first scheduling algorithm is derived from a Markov Decision Process (MDP)…
We consider a dynamic multichannel access problem, where multiple correlated channels follow an unknown joint Markov model. A user at each time slot selects a channel to transmit data and receives a reward based on the success or failure of…
In this paper, energy efficient power allocation for the uplink of a multi-cell massive MIMO system is investigated. With the simplified power consumption model, the problem of power allocation is formulated as a constrained Markov decision…
Energy Markov Decision Processes (EMDPs) are finite-state Markov decision processes where each transition is assigned an integer counter update and a rational payoff. An EMDP configuration is a pair s(n), where s is a control state and n is…
We study a dispatching and pricing problem in two-sided spatial queues with fixed supply, motivated by ride-hailing and robotaxi platforms. Idle drivers queue on one side, waiting to pick up riders, while riders queue on the other, waiting…
Routing in Delay-Tolerant Networks (DTNs) is inherently challenging due to sparse connectivity, long delays, and frequent disruptions. While Markov Decision Processes (MDPs) have been used to model uncertainty, they assume full state…
Serverless computing has emerged as a new execution model which gained a lot of attention in cloud computing thanks to the latest advances in containerization technologies. Recently, serverless has been adopted at the edge, where it can…
We consider stability and network capacity in discrete time queueing systems. Relationships between four common notions of stability are described. Specifically, we consider rate stability, mean rate stability, steady state stability, and…
Mobile micro-cloud is an emerging technology in distributed computing, which is aimed at providing seamless computing/data access to the edge of the network when a centralized service may suffer from poor connectivity and long latency.…
We present algorithms to effectively represent a set of Markov decision processes (MDPs), whose optimal policies have already been learned, by a smaller source subset for lifelong, policy-reuse-based transfer learning in reinforcement…
In many operations management problems, we need to make decisions sequentially to minimize the cost while satisfying certain constraints. One modeling approach to study such problems is constrained Markov decision process (CMDP). When…
Queue networks describe complex stochastic systems of both theoretical and practical interest. They provide the means to assess alterations, diagnose poor performance and evaluate robustness across sets of interconnected resources. In the…
We consider multiclass feedforward queueing networks with first in first out and priority service disciplines at the nodes, and class dependent deterministic routing between nodes. The random behavior of the network is constructed from…
The planning domain has experienced increased interest in the formal synthesis of decision-making policies. This formal synthesis typically entails finding a policy which satisfies formal specifications in the form of some well-defined…
In the context of railway systems, the application performance can be very critical and the radio conditions not advantageous. Hence, the communication problem parameters include both a survival time stemming from the application layer and…