Related papers: $M|G|\infty$ Queue Parameters Values Approximation…
In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…
Approximations for the mean performance indices for the M/G/c queue rely on the approximate computation of the probability that an arriving request has to wait for service and of the minimum of residual service times if all servers are…
In this paper we analyze an $M/M/1$ queueing system with an arbitrary number of customer classes, with class-dependent exponential service rates and preemptive priorities between classes. The queuing system can be described by a…
We study a model of a polling system, that is, a collection of $d$ queues with a single server that switches from queue to queue. The service time distribution and arrival rates change randomly every time a queue is emptied. This model is…
In this paper, we provide a novel algorithm for solving planning and learning problems of Markov decision processes. The proposed algorithm follows a policy iteration-type update by using a rank-one approximation of the transition…
Markov processes are used in a wide range of disciplines, including finance. The transition densities of these processes are often unknown. However, the conditional characteristic functions are more likely to be available, especially for…
In this note we consider M/D/1/N queue with renovation and derive analytic expressions for the following performance characteristics: stationary loss rate, moments of the number in the system. Moments of consecutive losses, waiting/sojourn…
Queueing networks are systems of theoretical interest that find widespread use in the performance evaluation of interconnected resources. In comparison to counterpart models in genetics or mathematical biology, the stochastic (jump)…
This paper presents a new condition for the existence of optimal stationary policies in average-cost continuous-time Markov decision processes with unbounded cost and transition rates, arising from controlled queueing systems. This…
Stochastic processes find applications in modelling systems in a variety of disciplines. A large number of stochastic models considered are Markovian in nature. It is often observed that higher order Markov processes can model the data…
We present a broad literature survey of parameter and state estimation for queueing systems. Our approach is based on various inference activities, queueing models, observations schemes, and statistical methods. We categorize these into…
We develop a qualitative theory of Markov Decision Processes (MDPs) and Partially Observable MDPs that can be used to model sequential decision making tasks when only qualitative information is available. Our approach is based upon an…
We consider Markov processes, which describe e.g. queueing network processes, in a random environment which influences the network by determining random breakdown of nodes, and the necessity of repair thereafter. Starting from an explicit…
We consider the convergence of a continuous-time Markov chain approximation X^h, h>0, to an R^d-valued Levy process X. The state space of X^h is an equidistant lattice and its Q-matrix is chosen to approximate the generator of X. In…
This paper calculates transient distributions of a special class of Markov processes with continuous state space and in continuous time, up to an explicit error bound. We approximate specific queues on R with one-sided L\'evy input, such as…
Queuing systems with an unlimited number of devices with an incoming nonstationary Poisson flow and a random flow controlled by a Markov chain are investigated. The inexpediency of ap-proximation of the birth process by Poisson flows in…
We propose a numerical technique for parameter inference in Markov models of biological processes. Based on time-series data of a process we estimate the kinetic rate constants by maximizing the likelihood of the data. The computation of…
In the Markov decision process model, policies are usually evaluated by expected cumulative rewards. As this decision criterion is not always suitable, we propose in this paper an algorithm for computing a policy optimal for the quantile…
Probabilistic model checking mainly concentrates on techniques for reasoning about the probabilities of certain path properties or expected values of certain random variables. For the quantitative system analysis, however, there is also…
We introduce and study some queueing models with random resetting, including Markovian and non--Markovian models. The Markovian models include M/M/$\infty$, M/M/r and M/M/1+M queues with random resetting, in which a continuous-time Markov…