Related papers: $M|G|\infty$ Queue Parameters Values Approximation…
We introduce the minimal maximally predictive models ({\epsilon}-machines) of processes generated by certain hidden semi-Markov models. Their causal states are either hybrid discrete-continuous or continuous random variables and…
The goal of a traditional Markov decision process (MDP) is to maximize expected cumulative reward over a defined horizon (possibly infinite). In many applications, however, a decision maker may be interested in optimizing a specific…
We consider a general method for the approximation of the distribution of a process conditioned to not hit a given set. Existing methods are based on particle system that are failable, in the sense that, in many situations , they are not…
Evaluating the expectation of a quantum circuit is a classically difficult problem known as the quantum mean value problem (QMV). It is used to optimize the quantum approximate optimization algorithm and other variational quantum…
The present paper provides some new stochastic inequalities for the characteristics of the $M/GI/1/n$ and $GI/M/1/n$ loss queueing systems. These stochastic inequalities are based on substantially deepen up- and down-crossings analysis, and…
A matching queue is described via a graph $G$ together with a matching policy. Specifically, to each node in the graph there is a corresponding arrival process of items which can either be queued, or matched with queued items in neighboring…
This paper presents a comprehensive review of stochastic processes, with a particular focus on Markov chains and jump processes. The main results related to queuing systems are analyzed. Additionally, conditions that ensure the stability,…
The mathematics of the finite single server queue with Poisson input and semi-Markov service times($M/SM/1/b$) is similar to that used for $BMAP/G/1/b$ systems. This observation results in new analytical formulas for a queue size in the…
We develop a martingale approximation approach to studying the limiting behavior of quadratic forms of Markov chains. We use the technique to examine the asymptotic behavior of lag-window estimators in time series and we apply the results…
Motion planning under uncertainty for an autonomous system can be formulated as a Markov Decision Process with a continuous state space. In this paper, we propose a novel solution to this decision-theoretic planning problem that directly…
We propose a method for approximating solutions to optimization problems involving the global stability properties of parameter-dependent continuous-time autonomous dynamical systems. The method relies on an approximation of the…
In this paper, we first use PDE techniques and probabilistic methods to identify a kind of quasi-continuous random variables. Then we give a characterization of the $G$-integrable processes and get a kind of quasi-continuous processes by…
In many cognitive radio applications, there are multiple types of message queues. Existing queueing analysis works in underlay CR networks failed to discuss packets heterogeneity. Therefore high priority packets with impatient waiting time…
In many important real-world queueing settings, arrival and service rates fluctuate over time. We consider the MAMS system, where the arrival and service rates each vary according to an arbitrary finite-state Markov chain, allowing…
Extensions of Kemeny's constant, as derived for irreducible finite Markov chains in discrete time, to Markov renewal processes and Markov chains in continuous time are discussed. Three alternative Kemeny's functions and their variants are…
This report considers a fairly general model of constrained queuing networks that allows us to represent both MMBP (Markov Modulated Bernoulli Processes) arrivals and time-varying service constraints. We derive a set of sufficient…
We consider the estimation of parameters encoded in the measurement record of a continuously monitored quantum system in the jump unraveling, corresponding to a single-shot scenario, where information is continuously gathered. Here, it is…
Value Iteration is a widely used algorithm for solving Markov Decision Processes (MDPs). While previous studies have extensively analyzed its convergence properties, they primarily focus on convergence with respect to the infinity norm. In…
In order to obtain Markov heavy-traffic approximations for infinite-server queues with general non-exponential service-time distributions and general arrival processes, possibly with time-varying arrival rates, we establish heavy-traffic…
We study a generalization of the $M/G/1$ system (denoted by $rM/G/1$) with independent and identically distributed (iid) service times and with an arrival process whose arrival rate $\lambda_0f(r)$ depends on the remaining service time $r$…