Related papers: Specific bounds for a probabilistically interpreta…
This paper studies continuous-time Markov decision processes under the risk-sensitive average cost criterion. The state space is a finite set, the action space is a Borel space, the cost and transition rates are bounded, and the…
We investigate an M/M/1 queue operating in two switching environments, where the switch is governed by a two-state time-homogeneous Markov chain. This model allows to describe a system that is subject to regular operating phases alternating…
We study the stochastic dynamics of a system of interacting species in a stochastic environment by means of a continuous-time Markov chain with transition rates depending on the state of the environment. Models of gene regulation in systems…
This note studies monotone Markov chains, a subclass of Markov chains with extensive applications in operations research and economics. While the properties that ensure the global stability of these chains are well studied, their…
We consider the first-crossing-time problem through a constant boundary for a Wiener process perturbed by random jumps driven by a counting process. On the base of a sample-path analysis of the jump-diffusion process we obtain explicit…
The Poisson boundary of a group G with a probability measure \mu is the space of ergodic components of the time shift in the path space of the associated random walk. Via a generalization of the classical Poisson formula it gives an…
Imprecise continuous-time Markov chains are a robust type of continuous-time Markov chains that allow for partially specified time-dependent parameters. Computing inferences for them requires the solution of a non-linear differential…
We derive uniform all-time concentration bound of the type 'for all $n \geq n_0$ for some $n_0$' for TD(0) with linear function approximation. We work with online TD learning with samples from a single sample path of the underlying Markov…
The purpose of this paper is to examine stochastic Markovian models for circuits in phase space for which the drift term is equivalent to the standard circuit equations. In particular we include dissipative components corresponding to both…
We consider a class of discrete time Markov chains with state space [0,1] and the following dynamics. At each time step, first the direction of the next transition is chosen at random with probability depending on the current location. Then…
We study the Markov chain on $\mathbf{F}_p$ obtained by applying a function $f$ and adding $\pm\gamma$ with equal probability. When $f$ is a linear function, this is the well-studied Chung--Diaconis--Graham process. We consider two cases:…
Consider a first-come, first-served single server queue with an initial workload $x>0$ and customers who arrive according to an inhomogeneous Poisson process with rate function $\lambda:[0,\infty)\rightarrow[0,\lambda_h ]$ for some…
We consider the compressible Poisson-Nernst-Planck-Navier-Stokes (PNPNS) system of equations, governing the transport of charged particles under the influence of the self-consistent electrostatic potential, in a three-dimensional bounded…
The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…
We study a generalized resolvent for the generator of a Markovian semigroup. The Markovian generator appears in a linear Boltzmann equation modeling a one-dimensional test particle in a periodic potential and colliding elastically with…
This paper deals with U-statistics of Poisson processes and multiple Wiener-It\^o integrals on the Poisson space. Via sharp bounds on the cumulants for both classes of random variables, moderate deviation principles, concentration…
We study spherically symmetric solutions of the Vlasov-Poisson system in the context of algebras of generalized functions. This allows to model highly concentrated initial configurations and provides a consistent setting for studying…
In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…
This work provides complete description of Quasistationary Distributions (QSDs) for Markov chains with a unique absorbing state and an irreducible set of non-absorbing states. As is well-known, every QSD has an associated absorption…
We address a class of Markov jump linear systems that are characterized by the underlying Markov process being time-inhomogeneous with a priori unknown transition probabilities. Necessary and sufficient conditions for uniform stochastic…