Related papers: $M|G|\infty$ Queue Parameters Values Approximation…
Computing reachability probabilities is at the heart of probabilistic model checking. All model checkers compute these probabilities in an iterative fashion using value iteration. This technique approximates a fixed point from below by…
The paper analyzes the performance of tandem network of polling queue with setups. For a system with two-products and two-stations, we propose a new approach based on a partially-collapsible state-space characterization to reduce…
Novel advanced policy gradient (APG) methods, such as Trust Region policy optimization and Proximal policy optimization (PPO), have become the dominant reinforcement learning algorithms because of their ease of implementation and good…
We consider Markov decision processes where the state of the chain is only given at chosen observation times and of a cost. Optimal strategies involve the optimisation of observation times as well as the subsequent action values. We…
Supermarket models with different servers become a key in modeling resource management of stochastic networks, such as, computer networks, manufacturing systems and transportation networks. While these different servers always make analysis…
Estimating properties of a quantum state is an indispensable task in various applications of quantum information processing. To predict properties in the post-processing stage, it is inherent to first perceive the quantum state with a…
We consider the problem of approximating the reachability probabilities in Markov decision processes (MDP) with uncountable (continuous) state and action spaces. While there are algorithms that, for special classes of such MDP, provide a…
This article deals with stochastic processes endowed with the Markov (memoryless) property and evolving over general (uncountable) state spaces. The models further depend on a non-deterministic quantity in the form of a control input, which…
In this paper we study a non-stationary Markovian queueing model of a two-processor heterogeneous system with time-varying arrival and service rates. We obtain the bounds on the rate of convergence and find the main limiting characteristics…
In this paper, we consider a queueing network with $N$ nodes, each of which has a fixed number $k$ of neighboring nodes, referred to as the $N$ node network with local balancing. We assume that to each of the $N$ nodes, an incoming job (or…
We consider an $M/M/1$ queueing system with impatient customers with multiple and single vacations. It is assumed that customers are impatient whenever the state of the server. We derive the probability generating functions of the number of…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
This paper presents a Bayesian method for identification of jump Markov linear system parameters. A primary motivation is to provide accurate quantification of parameter uncertainty without relying on asymptotic in data-length arguments. To…
We investigate an M/M/1 queue operating in two switching environments, where the switch is governed by a two-state time-homogeneous Markov chain. This model allows to describe a system that is subject to regular operating phases alternating…
This document presents a compilation of results related to the theory of stochastic processes, with a specific focus on Markov processes, regenerative processes, renewal processes, and stationary processes. The relevance of these topics…
We consider large-scale Markov decision processes (MDPs) with a risk measure of variability in cost, under the risk-aware MDPs paradigm. Previous studies showed that risk-aware MDPs, based on a minimax approach to handling risk, can be…
Through the analysis of a dataset of ultra high frequency order book updates, we introduce a model which accommodates the empirical properties of the full order book together with the stylized facts of lower frequency financial data. To do…
This article examines the queueing performance of communication systems that transmit encoded data over unreliable channels. A fading formulation suitable for wireless environments is considered where errors are caused by a discrete channel…
New recursive equations designed for the G/G/m queue are presented. These equations describe the queue in terms of recursions for the arrival and departure times of customers, and involve only the operations of maximum, minimum and…
In complex systems, it is quite common to resort to approximations when optimizing system performance. These approximations typically involve selecting a particular system parameter and then studying the performance of the system as this…