Related papers: Performance measures for the two-node queue with f…
We develop a neural-network framework for multi-period risk--reward stochastic control problems with constrained two-step feedback policies that may be discontinuous in the state. We allow a broad class of objectives built on a…
A heterogeneous closed network with one-server queues with finite capacity and one infinite-server queue is studied. A target application is bike-sharing systems. Heterogeneity is taken into account through clusters whose queues have the…
Suppose an online platform wants to compare a treatment and control policy, e.g., two different matching algorithms in a ridesharing system, or two different inventory management algorithms in an online retail site. Standard randomized…
In this paper one presents method for the computation of convergence bounds for four classes of multiserver queueing systems, described by inhomogeneous Markov chains. Specifically one considers inhomogeneous $M/M/S$ queueing system with…
Some $M|G|\infty$ queue systems parameters values approximations, obtained through the consideration of an adequate Markov renewal process, are presented, and studied.
The models studied in the steady state involve two queues which are served either by a single server whose speed depends on the number of jobs present, or by several parallel servers whose number may be controlled dynamically. Job service…
We explore the use of machine-learning techniques to detect quantum speedup in random walks on graphs. Specifically, we investigate the performance of three different neural-network architectures (variations on fully connected and…
Random walks are a fundamental model in applied mathematics and are a common example of a Markov chain. The limiting stationary distribution of the Markov chain represents the fraction of the time spent in each state during the stochastic…
The problem of exact evaluation of the mean service cycle time in tandem systems of single-server queues with both infinite and finite buffers is considered. It is assumed that the interarrival and service times of customers form sequences…
For a Markov chain $Y$ with values in a Polish space, consider the entrance chain, obtained by sampling $Y$ at the moments when it enters a fixed set $A$ from its complement $A^c$. Similarly, consider the exit chain, obtained by sampling…
This paper considers the queueing performance of a system that transmits coded data over a time-varying erasure channel. In our model, the queue length and channel state together form a Markov chain that depends on the system parameters.…
We apply the method of differential inequalities for the computation of upper bounds for the rate of convergence to the limiting regime for one specific class of (in)homogeneous continuous-time Markov chains. To obtain these estimates, we…
We present a new algorithm for the statistical model checking of Markov chains with respect to unbounded temporal properties, such as reachability and full linear temporal logic. The main idea is that we monitor each simulation run on the…
The treatment of two-dimensional random walks in the quarter plane leads to Markov processes which involve semi-infinite matrices having Toeplitz or block Toeplitz structure plus a low-rank correction. Finding the steady state probability…
The contribution gives a micro-structural insight into the pedestrian decision process during an egress situation. A method how to extract the decisions of pedestrians from the trajectories recorded during the experiments is introduced. The…
We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by "controlled" Markov noise. In particular, the faster and slower recursions have non-additive controlled Markov noise…
A one-dimensional confined Nonlinear Random Walk is a tuple of $N$ diffeomorphisms of the unit interval driven by a probabilistic Markov chain. For generic such walks, we obtain a geometric characterization of their ergodic stationary…
We study the problem of detecting a random walk on a graph from a sequence of noisy measurements at every node. There are two hypotheses: either every observation is just meaningless zero-mean Gaussian noise, or at each time step exactly…
We study two time-scale linear stochastic approximation algorithms, which can be used to model well-known reinforcement learning algorithms such as GTD, GTD2, and TDC. We present finite-time performance bounds for the case where the…
We consider random walks in which the walk originates in one set of nodes and then continues until it reaches one or more nodes in a target set. The time required for the walk to reach the target set is of interest in understanding the…