Related papers: Perfect simulation of Markovian load balancing que…
Standard Markovian optimal stopping problems are consistent in the sense that the first entrance time into the stopping set is optimal for each initial state of the process. Clearly, the usual concept of optimality cannot in a…
We consider a parallel system of $m$ identical machines prone to unpredictable crashes and restarts, trying to cope with the continuous arrival of tasks to be executed. Tasks have different computational requirements (i.e., processing time…
This paper first describes a class of uncertain stochastic control systems with Markovian switching, and derives an It\^o-Liu formula for Markov-modulated processes. And we characterize an optimal control law, which satisfies the…
We propose a comprehensive framework for policy gradient methods tailored to continuous time reinforcement learning. This is based on the connection between stochastic control problems and randomised problems, enabling applications across…
A novel class of non-reversible Markov chain Monte Carlo schemes relying on continuous-time piecewise-deterministic Markov Processes has recently emerged. In these algorithms, the state of the Markov process evolves according to a…
In this paper we investigate the continuum limits of a class of Markov chains. The investigation of such limits is motivated by the desire to model very large networks. We show that under some conditions, a sequence of Markov chains…
Quantum Monte-Carlo simulations of hybrid quantum-classical models such as the double exchange Hamiltonian require calculating the density of states of the quantum degrees of freedom at every step. Unfortunately, the computational…
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…
In this paper, we introduce a slight variation of the Dominated Coupling From the Past algorithm (DCFTP) of Kendall, for bounded Markov chains. It is based on the control of a (typically non-monotonic) stochastic recursion by a (typically…
In this paper, we propose a new policy iteration algorithm to compute the value function and the optimal controls of continuous time stochastic control problems. The algorithm relies on successive approximations using linear-quadratic…
We consider the problem of flexible modeling of higher order hidden Markov models when the number of latent states and the nature of the serial dependence, including the true order, are unknown. We propose Bayesian nonparametric methodology…
We introduce a new class of combinatorial markets in which agents have covering constraints over resources required and are interested in delay minimization. Our market model is applicable to several settings including scheduling, cloud…
Motivated by time-sensitive e-service applications, we consider the design of effective policies in a Markovian model for the dynamic control of both admission and routing of a single class of real-time transactions to multiple…
We consider the problem of allocating indivisible goods in a way that is fair, using one of the leading market mechanisms in economics: the competitive equilibrium from equal incomes. Focusing on two major classes of valuations, namely…
Classical queuing network control strategies typically rely on accurate knowledge of model data, i.e., arrival and service rates. However, such data are not always available and may be time-variant. To address this challenge, we consider a…
The Multi-class Queueing Network (McQN) arises as a natural multi-class extension of the traditional (single-class) Jackson network. In a single-class network subcriticality (i.e. subunitary nominal workload at every station) entails…
Markov decision processes continue to gain in popularity for modeling a wide range of applications ranging from analysis of supply chains and queuing networks to cognitive science and control of autonomous vehicles. Nonetheless, they tend…
Inspired by quantum switches, we consider a discrete-time multi-way matching system with two classes of arrivals: requests for entangled pair of qubits between two nodes, and qubits from each node that can be used to serve the requests. An…
In this paper we show that in a multiclass Markovian network with unit rate servers, the condition that the average load $\rho$ at every server is less than unity is indeed sufficient for the stability or positive recurrence for \emph{any}…
Advances in sampling schemes for Markov jump processes have recently enabled multiple inferential tasks. However, in statistical and machine learning applications, we often require that these continuous-time models find support on…