Related papers: Performance measures for the two-node queue with f…
Predictability of behavior has emerged an an important characteristic in many fields including biology, medicine, and marketing. Behavior can be recorded as a sequence of actions performed by an individual over a given time period. This…
We study random walk on complex networks with transition probabilities which depend on the current and previously visited nodes. By using an absorbing Markov chain we derive an exact expression for the mean first passage time between pairs…
This paper focuses on the numerical approximation of random lattice reversible Selkov systems. It establishes the existence of numerical invariant measures for random models with nonlinear noise, using the backward Euler-Maruyama (BEM)…
A multi-class single-server queueing model with finite buffers, in which scheduling and admission of customers are subject to control, is studied in the moderate deviation heavy traffic regime. A risk-sensitive cost set over a finite time…
Parallel algorithms designed for simulation and performance evaluation of single-server tandem queueing systems with both infinite and finite buffers are presented. The algorithms exploit a simple computational procedure based on recursive…
In this paper we study the asymptotic behavior of the Random-Walk Metropolis algorithm on probability densities with two different `scales', where most of the probability mass is distributed along certain key directions with the…
We present a novel strategy for robust dual control of linear time-invariant systems based on gain scheduling with performance guarantees. This work relies on prior results of determining uncertainty bounds of system parameters estimated…
We consider the problem of detecting a random walk on a graph, based on observations of the graph nodes. When visited by the walk, each node of the graph observes a signal of elevated mean, which we assume can be different across different…
A notorious problem in queueing theory is to compute the worst possible performance of the GI/G/1 queue under mean-dispersion constraints for the interarrival and service time distributions. We address this extremal queue problem by…
Discrete Markov random fields form a natural class of models to represent images and spatial data sets. The use of such models is, however, hampered by a computationally intractable normalising constant. This makes parameter estimation and…
We study the policy evaluation problem in multi-agent reinforcement learning, modeled by a Markov decision process. In this problem, the agents operate in a common environment under a fixed control policy, working together to discover the…
We consider a two-node tandem queueing network in which the upstream queue is M/G/1 and each job reuses its upstream service requirement when moving to the downstream queue. Both servers employ the first-in-first-out policy. We investigate…
In the paper, the approximate sequence for entropy of some binary hidden Markov models has been found to have two bound sequences, the low bound sequence and the upper bound sequence. The error bias of the approximate sequence is bound by a…
Consider an M/M/$s$ queue with the additional feature that the arrival rate is a random variable of which only the mean, variance, and range are known. Using semi-infinite linear programming and duality theory for moment problems, we…
We present a comparative study of several algorithms for an in-plane random walk with a variable step. The goal is to check the efficiency of the algorithm in the case where the random walk terminates at some boundary. We recently found…
Perturbation analysis of Markov chains provides bounds on the effect that a change in a Markov transition matrix has on the corresponding stationary distribution. This paper compares and analyzes bounds found in the literature for finite…
We study time-inhomogeneous Markov chains to obtain quantitative results on their asymptotic behavior. We use Poincar\'e, Nash, and logarithmic-Sobolev inequalities. We assume that our Markov chain admits a finite invariant measure at each…
We consider a discrete-time Markov decision process with Borel state and action spaces. The performance criterion is to maximize a total expected {utility determined by unbounded return function. It is shown the existence of optimal…
Sequential scaling is a prominent inference-time scaling paradigm, yet its performance improvements are typically modest and not well understood, largely due to the prevalence of heuristic, non-principled approaches that obscure clear…
For a random walk defined for a doubly infinite sequence of times, we let the time parameter itself be an integer-valued process, and call the orginal process a random walk at random time. We find the scaling limit which generalizes the…