Related papers: Reward Processes and Performance Simulation in Sup…
In queuing theory and related problems, it is very important to know the numerical characteristics of an investigated system - both in stationary and non-stationary modes. In some cases, such characteristics can be calculated, but this is…
Demand for studying queueing systems with multiple servers providing correlated services was created about 60 years ago, motivated by various applications. In recent years, the importance of such studies has been significantly increased,…
Queue networks describe complex stochastic systems of both theoretical and practical interest. They provide the means to assess alterations, diagnose poor performance and evaluate robustness across sets of interconnected resources. In the…
In this paper the infinite server queue model in semi-Markov random environment with k Markov arrival streams, random resources of customers, and catastrophes is considered. After catastrophes occur, all customers in the model are flashed…
Frameworks, such as MapReduce and Hadoop are abundant nowadays. They seek to reap benefits of parallelization, albeit subject to a synchronization constraint at the output. Fork-Join (FJ) queuing models are used to analyze such systems.…
New algorithms for computing power moments of hitting times and accumulated rewards of hitting type for semi-Markov processes. The algorithms are based on special techniques of sequential phase space reduction and recurrence relations…
A Markovian single-server queue is studied in an interactive random environment. The arrival and service rates of the queue depend on the environment, while the transition dynamics of the random environment depends on the queue length. We…
A FORTRAN program to simulate the operation of infinite servers queues is presented in this work. Poisson arrivals processes are considered but not only. For many parameters of interest in queuing systems study or application, either there…
We are interested in the analysis of very large continuous-time Markov chains (CTMCs) with many distinct rates. Such models arise naturally in the context of reliability analysis, e.g., of computer network performability analysis, of power…
We consider a stochastic, dynamic job scheduling problem, formulated as a queueing control problem, in which a single server processes jobs of different types that arrive according to independent Poisson processes. The problem is defined on…
Randomized load balancing networks arise in a variety of applications, and allow for efficient sharing of resources, while being relatively easy to implement. We consider a network of parallel queues in which incoming jobs with independent…
In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…
We study and develop the stochastic Markov reward model (sMRM), which extends the Markov chain where transition time/reward as modelled as random variables. Techniques are presented to enable computing first-passage time distributions (or…
Markov models are often used to capture the temporal patterns of sequential data for statistical learning applications. While the Hidden Markov modeling-based learning mechanisms are well studied in literature, we analyze a…
We study the relation between different Markov Decision Process (MDP) frameworks in the machine learning and econometrics literatures, including the standard MDP, the entropy and general regularized MDP, and stochastic MDP, where the latter…
We consider the problem of multiple sensor scheduling for remote state estimation of multiple process over a shared link. In this problem, a set of sensors monitor mutually independent dynamical systems in parallel but only one sensor can…
Motivated by queueing systems with heterogeneous parallel servers, we consider a class of structured multi-dimensional Markov processes whose state space can be partitioned into two parts: a finite set of boundary states and a structured…
In this paper we analyze an $M/M/1$ queueing system with an arbitrary number of customer classes, with class-dependent exponential service rates and preemptive priorities between classes. The queuing system can be described by a…
Process mining is a well-established discipline of data analysis focused on the discovery of process models from information systems' event logs. Recently, an emerging subarea of process mining, known as stochastic process discovery, has…
We investigate Markovian queues that are examined by a controller at random times determined by a Poisson process. Upon examination, the controller sets the service speed to be equal to the minimum of the current number of customers in the…