Related papers: Stochastic Optimization for Markov Modulated Netwo…
This dissertation is a study on the design and analysis of novel, optimal routing and rate control algorithms in wireless, mobile communication networks. Congestion control and routing algorithms upto now have been designed and optimized…
Stochastic network optimization problems entail finding resource allocation policies that are optimum on an average but must be designed in an online fashion. Such problems are ubiquitous in communication networks, where resources such as…
Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…
In this paper, we study the delay-constrained input-queued switch where each packet has a deadline and it will expire if it is not delivered before its deadline. Such new scenario is motivated by the proliferation of real-time applications…
This work considers the sensor scheduling for multiple dynamic processes. We consider $n$ linear dynamic processes, the state of each process is measured by a sensor, which transmits their local state estimates over wireless channels to a…
We consider cross-layer design of delay optimal transmission strategies for energy harvesting transmitters where the data and energy arrival processes are stochastic. Using Markov decision theory, we show that the value function is weakly…
In this paper, delay-optimal and energy-efficient communication is studied for a single link under Markov random arrivals. We present the optimal tradeoff between delay and power over Additive White Gaussian Noise (AWGN) channels and extend…
This paper proposes a dynamic primal-dual type algorithm to solve the optimal scheduling problem in wireless networks subject to uncertain parameters, which are generated by stochastic network processes such as random packet arrivals,…
In recent years, offloading mobile traffic through Wi-Fi has emerged as a potential solution to lower down the communication cost for mobile users. Users hope to reduce the cost while keeping the delay in an acceptable range through Wi-Fi…
Caching and multicasting at base stations are two promising approaches to support massive content delivery over wireless networks. However, existing scheduling designs do not make full use of the advantages of the two approaches. In this…
We consider a discrete time stochastic queueing system where a controller makes a 2-stage decision every slot. The decision at the first stage reveals a hidden source of randomness with a control-dependent (but unknown) probability…
Recently, low-complexity and distributed Carrier Sense Multiple Access (CSMA)-based scheduling algorithms have attracted extensive interest due to their throughput-optimal characteristics in general network topologies. However, these…
A trade-off between two QoS requirements of wireless sensor networks: query waiting time and validity (age) of the data feeding the queries, is investigated. We propose a Continuous Time Markov Decision Process with a drift that trades-off…
In this paper, we consider delay-optimal power and subcarrier allocation design for OFDMA systems with $N_F$ subcarriers, $K$ mobiles and one base station. There are $K$ queues at the base station for the downlink traffic to the $K$ mobiles…
Optimal sensor scheduling with applications to networked estimation and control systems is considered. We model sensor measurement and transmission instances using jumps between states of a continuous-time Markov chain. We introduce a cost…
In this paper, we investigate a novel minimum length scheduling problem to determine the optimal power control, and scheduling for constant and continuous rate models, while considering concurrent transmission of users, energy causality,…
In this paper, we aim to obtain the optimal delay-power tradeoff and the corresponding optimal scheduling policy for an arbitrary i.i.d. arrival process and adaptive transmissions. The number of backlogged packets at the transmitter is…
The problem of constrained Markov decision process is considered. An agent aims to maximize the expected accumulated discounted reward subject to multiple constraints on its costs (the number of constraints is relatively small). A new dual…
In this paper, we propose a systematic solution to the problem of cross-layer optimization for delay-sensitive media transmission over time-varying wireless channels as well as investigate the structures and properties of this solution,…
In this paper, we consider the dynamic power control for delay-aware D2D communications. The stochastic optimization problem is formulated as an infinite horizon average cost Markov decision process. To deal with the curse of…