Related papers: Sample Path Moderate Deviation Principle for Queue…
We introduce and analyze Markov Decision Process (MDP) machines to model individual devices which are expected to participate in future demand-response markets on distribution grids. We differentiate devices into the following four types:…
This paper provides conditions under which total-cost and average-cost Markov decision processes (MDPs) can be reduced to discounted ones. Results are given for transient total-cost MDPs with tran- sition rates whose values may be greater…
The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…
We investigate the transient and stationary queue-length distributions of a class of service systems with correlated service times. The classical $M^X/G/1$ queue with semi-Markov service times is the most prominent example in this class and…
Fairness has emerged as an important concern in automated decision-making in recent years, especially when these decisions affect human welfare. In this work, we study fairness in temporally extended decision-making settings, specifically…
We consider a collection of fully coupled weakly interacting diffusion processes moving in a two-scale environment. We study the moderate deviations principle of the empirical distribution of the particles' positions in the combined limit…
A many-server queueing system is considered in which customers arrive according to a renewal process and have service and patience times that are drawn from two independent sequences of independent, identically distributed random variables.…
We consider multiple parallel Markov decision processes (MDPs) coupled by global constraints, where the time varying objective and constraint functions can only be observed after the decision is made. Special attention is given to how well…
A new class of stochastic processes called independent and periodically identically distributed (i.p.i.d.) processes is defined to capture periodically varying statistical behavior. A novel Bayesian theory is developed for detecting a…
We study a multi-server queueing system with a periodic arrival rate and customers whose joining decision is based on their patience and a delay proxy. Specifically, each customer has a patience level sampled from a common distribution.…
The incidence of rare events in fast-slow systems is investigated via analysis of the large deviation principle (LDP) that characterizes the likelihood and pathway of large fluctuations of the slow variables away from their mean behavior --…
The model is a "generalized switch", serving multiple traffic flows in discrete time. The switch uses MaxWeight algorithm to make a service decision (scheduling choice) at each time step, which determines the probability distribution of the…
For the M/M/1+M model at the law-of-large-numbers scale, the long run reneging count per unit time does not depend on the individual (i.e., per customer) reneging rate. This paradoxical statement has a simple proof. Less obvious is a large…
In this paper, we study the averaging principle and central limit theorem for multi-scale stochastic differential equations with state-dependent switching. To accomplish this, we first study the Poisson equation associated with a Markov…
In this paper, we study small noise asymptotics of Markov-modulated diffusion processes in the regime that the modulating Markov chain is rapidly switching. We prove the joint sample-path large deviations principle for the Markov-modulated…
Labeled continuous-time Markov chains (CTMCs) describe processes subject to random timing and partial observability. In applications such as runtime monitoring, we must incorporate past observations. The timing of these observations matters…
Interval Markov decision processes are a class of Markov models where the transition probabilities between the states belong to intervals. In this paper, we study the problem of efficient estimation of the optimal policies in Interval…
A moderate deviation principle for functionals, with at most quadratic growth, of moving average processes is established. The main assumptions on the moving average process are a Logarithmic Sobolev inequality for the driving random…
Piecewise-deterministic Markov processes (PDMPs) offer a powerful stochastic modeling framework that combines deterministic trajectories with random perturbations at random times. Estimating their local characteristics (particularly the…
We consider a Markovian many server queueing system in which customers are preemptively scheduled according to exogenously assigned priority levels. The priority levels are randomly assigned from a continuous probability measure rather than…