Related papers: Performance measures for the two-node queue with f…
This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization…
We are interested in a large queue in a $GI/G/k$ queue with heterogeneous servers. For this, we consider tail asymptotics and weak limit approximations for the stationary distribution of its queue length process in continuous time under a…
In recent years probabilistic model checking has become an important area of research because of the diffusion of computational systems of stochastic nature. Despite its great success, standard probabilistic model checking suffers the…
For a Markov chain $Y$ with values in a Polish space, consider the entrance Markov chain obtained by sampling $Y$ at the moments when it enters a fixed set $A$ from its complement $A^c$. Similarly, consider the exit Markov chain, obtained…
We consider a Markov chain $(x_n)$ whose kernel is indexed by a scaling parameter $\gamma>0$, refered to as the step size. The aim is to analyze the behavior of the Markov chain in the doubly asymptotic regime where $n\to\infty$ then…
We investigate the quantum versions of a one-dimensional random walk, whose corresponding Markov Chain is of order 2. This corresponds to the walk having a memory of up to two previous steps. We derive the amplitudes and probabilities for…
This paper studies function approximation for finite horizon discrete time Markov decision processes under certain convexity assumptions. Uniform convergence of these approximations on compact sets is proved under several sampling schemes…
In this paper we consider the statistics of repeated measurements on the output of a quantum Markov chain. We establish a large deviations result analogous to Sanov's theorem for the empirical measure associated to finite sequences of…
Available throughput performance studies for mobile ad hoc networks (MANETs) suffer from two major limitations: they mainly focus on the scaling law study of throughput, while the exact throughput of such networks remains largely unknown;…
Bisimulation metrics are powerful tools for measuring similarities between stochastic processes, and specifically Markov chains. Recent advances have uncovered that bisimulation metrics are, in fact, optimal-transport distances, which has…
The stochastic growth-fragmentation model describes the temporal evolution of a structured cell population through a discrete-time and continuous-state Markov chain. The simulations of this stochastic process and its invariant measure are…
Biochemical reactions can happen on different time scales and also the abundance of species in these reactions can be very different from each other. Classical approaches, such as deterministic or stochastic approach, fail to account for or…
Optimal sensor scheduling with applications to networked estimation and control systems is considered. We model sensor measurement and transmission instances using jumps between states of a continuous-time Markov chain. We introduce a cost…
We develop a martingale approximation approach to studying the limiting behavior of quadratic forms of Markov chains. We use the technique to examine the asymptotic behavior of lag-window estimators in time series and we apply the results…
We study networks of interacting queues governed by utility-maximising service-rate allocations in both discrete and continuous time. For {\em finite} networks we establish stability and some steady-state moment bounds under natural…
We are interested in the analysis of very large continuous-time Markov chains (CTMCs) with many distinct rates. Such models arise naturally in the context of reliability analysis, e.g., of computer network performability analysis, of power…
Motivated by applications in online marketplaces such as ride-hailing platforms and payment channel networks, we study a single-server queue with state-dependent arrival control. The service operator dynamically chooses the arrival rate as…
To ensure the effective and objective development of transportation networks, it is crucial to identify performance limitations across various subsystems. A timetable-independent assessment of infrastructure capacity at railway junctions is…
We demonstrate the use of a variational method to determine a quantitative lower bound on the rate of convergence of Markov Chain Monte Carlo (MCMC) algorithms as a function of the target density and proposal density. The bound relies on…
This paper considers a particular renewal-reward process with multivariate discounted rewards (inputs) where the arrival epochs are adjusted by adding some random delays. Then this accumulated reward can be regarded as multivariate…