Related papers: $M|G|\infty$ Queue Parameters Values Approximation…
We consider finite horizon Markov decision processes under performance measures that involve both the mean and the variance of the cumulative reward. We show that either randomized or history-based policies can improve performance. We prove…
The paper provides an overview of the theory and applications of risk-sensitive Markov decision processes. The term 'risk-sensitive' refers here to the use of the Optimized Certainty Equivalent as a means to measure expectation and risk.…
Tandem queueing systems are widely-used stochastic models that arise from many real-life service operations systems. Motivated by the desire to understand the trade-off between the performance and complexity of policies for…
In this paper we introduce and study renewal-reward processes in random environments where each renewal involves a reward taking values in a Banach space. We derive quenched large deviation principles and identify the associated rate…
Markov decision processes (MDPs) are a popular model for performance analysis and optimization of stochastic systems. The parameters of stochastic behavior of MDPs are estimates from empirical observations of a system; their values are not…
We are interested in studying the sensitivity of diffusion processes or their approximations by Markov Chains with respect to a perturbation of the coefficients.
In Markov decision processes (MDPs), quantile risk measures such as Value-at-Risk are a standard metric for modeling RL agents' preferences for certain outcomes. This paper proposes a new Q-learning algorithm for quantile optimization in…
We consider the problem of estimating expectations by using Markov chain Monte Carlo methods and improving the accuracy by replacing IID uniform random points with quasi-Monte Carlo (QMC) points. Recently, it has been shown that Markov…
Queueing-inventory systems are integrated systems consisting of two emerging fields in applied probability, namely "Queues" and "Inventory". In this paper, we present a comprehensive review of the theory and applications of…
For Markov processes with absorption, we provide general criteria ensuring the existence and the exponential non-uniform convergence in total variation norm to a quasi-stationary distribution. We also characterize a subset of its domain of…
We present a new method for statistical verification of quantitative properties over a partially unknown system with actions, utilising a parameterised model (in this work, a parametric Markov decision process) and data collected from…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…
We study a system, where a random flow of customers is served by servers (called agents) invited on-demand. Each invited agent arrives into the system after a random time; after each service completion, an agent returns to the system or…
We introduce a simple method to estimate the system parameters in continuous dynamical systems from the time series. In this method, we construct a modified system by introducing some constants (controlling constants) into the given…
A multi-channel queueing system is considered. The arriving requests differ in their type. Requests of each type arrive according to a Poisson process. The number of channels required for service with the rate equal to 1 depends of the…
In this paper we propose a new method for approximating the nonstationary moment dynamics of one dimensional Markovian birth-death processes. By expanding the transition probabilities of the Markov process in terms of Poisson-Charlier…
We consider the down/up crossing property of weighted Markov branching processes. The joint probability distribution of multi crossing numbers of such processes are obtained. In particular, for Markov branching processes, the probability…
We describe an exact approach for calculating transition probabilities and waiting times in finite-state discrete-time Markov processes. All the states and the rules for transitions between them must be known in advance. We can then…
A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be…
Continuous value prediction plays a crucial role in industrial-scale recommendation systems, including tasks such as predicting users' watch-time and estimating the gross merchandise value (GMV) in e-commerce transactions. However, it…