Related papers: Stationary Markovian Arrival Processes, Results an…
This paper studies the exponential stability of random matrix products driven by a general (possibly unbounded) state space Markov chain. It is a cornerstone in the analysis of stochastic algorithms in machine learning (e.g. for parameter…
We investigate stationary hidden Markov processes for which mutual information between the past and the future is infinite. It is assumed that the number of observable states is finite and the number of hidden states is countably infinite.…
This paper investigates natural conditions for the existence of optimal policies for a Markov decision process with incomplete information (MDPII) and with expected total costs. The MDPII is the classic model of a controlled stochastic…
This paper addresses the challenge of a particular class of noisy state observations in Markov Decision Processes (MDPs), a common issue in various real-world applications. We focus on modeling this uncertainty through a confusion matrix…
It was recently proven that the correlation function of the stationary version of a reflected L\'evy process is nonnegative, nonincreasing and convex. In another branch of the literature it was established that the mean value of the…
We introduce stochastic decision Petri nets (SDPNs), which are a form of stochastic Petri nets equipped with rewards and a control mechanism via the deactivation of controllable transitions. Such nets can be translated into Markov decision…
We provide a general framework for dual representations of Laplace transforms of Markov processes. Such representations state that the Laplace transform of a finite-dimensional distribution of a Markov process can be expressed in terms of a…
By modeling the interaction of a system with an environment through a renewal approach, we demonstrate that completely positive non-Markovian dynamics may develop some unexplored non-standard statistical properties. The renewal approach is…
We consider a stationary Markovian evolution with values on a disjointly partitioned set space $I\sqcup {\cal E}$. The evolution is visible (in the sense of knowing the transition probabilities) on the states in $I$ but not for the states…
Ba\~nuelos and Bogdan (2004) and Bogdan, Palmowski and Wang (2016) analyse the asymptotic tail distribution of the first time a stable (L\'evy) process in dimension $d\geq 2$ exists a cone. We use these results to develop the notion of a…
The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant "additivity" properties: (i) existence of a…
Markov decision processes (MDPs) are a popular model for performance analysis and optimization of stochastic systems. The parameters of stochastic behavior of MDPs are estimates from empirical observations of a system; their values are not…
Stochastic exclusion processes play an integral role in the physics of non-equilibrium statistical mechanics. These models are Markovian processes, described by a classical master equation. In this paper a quantum mechanical version of a…
We study a class of systems termed Markov Machines (MM) which process job requests with exponential service times. Assuming a Poison job arrival process, these MMs oscillate between two states, free and busy. We consider the problem of…
Let {X_n,n\geq0} be a Markov chain on a general state space X with transition probability P and stationary probability \pi. Suppose an additive component S_n takes values in the real line R and is adjoined to the chain such that…
We introduce an extension of the M/M/1 queueing process with a spatial structure and excluded- volume effect. The rule of particle hopping is the same as for the totally asymmetric simple exclusion process (TASEP). A stationary-state…
Structural results impose sufficient conditions on the model parameters of a Markov decision process (MDP) so that the optimal policy is an increasing function of the underlying state. The classical assumptions for MDP structural results…
Reinforcement learning in non-stationary environments is challenging due to abrupt and unpredictable changes in dynamics, often causing traditional algorithms to fail to converge. However, in many real-world cases, non-stationarity has some…
Symmetric determinantal point processes (DPP's) are a class of probabilistic models that encode the random selection of items that exhibit a repulsive behavior. They have attracted a lot of attention in machine learning, when returning…
We study ergodic properties of Markovian multiclass many-server queues which are uniform over scheduling policies, as well as the size n of the system. The system is heavily loaded in the Halfin-Whitt regime, and the scheduling policies are…