Related papers: Distributed Algorithms in an Ergodic Markovian Env…
We provide a large deviations analysis of deadlock phenomena occurring in distributed systems sharing common resources. In our model transition probabilities of resource allocation and deallocation are time and space dependent. The process…
We consider the time dependent probability distribution of a coarse grained observable Y whose evolution is governed by a discrete time map. If the map is mixing, the time dependent one-step transition probabilities converge in the long…
We study the ergodic behaviour of a discrete-time process $X$ which is a Markov chain in a stationary random environment. The laws of $X_t$ are shown to converge to a limiting law in (weighted) total variation distance as $t\to\infty$.…
We consider continuous-time random walk models described by arbitrary sojourn time probability density functions. We find a general expression for the distribution of time-averaged observables for such systems, generalizing some recent…
Probabilistic timed automata are classical timed automata extended with discrete probability distributions over edges. We introduce clock-dependent probabilistic timed automata, a variant of probabilistic timed automata in which transition…
The purpose of this paper is to study the time average behavior of Markov chains with transition probabilities being kernels of completely continuous operators, and therefore to provide a sufficient condition for a class of Markov chains…
We study time-changed Markov processes to speed up the convergence of Markov chain Monte Carlo (MCMC) algorithms. The time-changed process is defined by adjusting the speed of time of a base process via a user-chosen, state-dependent…
We study the optimization of ergodic averages for multi-valued dynamical systems, i.e. where points may have multiple different forward orbits. Under upper semi-continuity assumptions, we show that the maximum space average with respect to…
A study of time homogeneous, real valued Markov processes with a special property and a non-atomic initial distribution is provided. The new notion of a function of evolution of distribution which determines the dependency between one…
In this paper, we consider semi-Markov processes whose transition times and transition probabilities depend on a small parameter $\varepsilon$. Understanding the asymptotic behavior of such processes is needed in order to study the…
We consider a continuous time Markov process on $\mathbb{N}_0$ which can be interpreted as generalized alternating birth-death process in a non-autonomous random environment. Depending on the status of the environment the process either…
We study a general class of random walks driven by a uniquely ergodic Markovian environment. Under a coupling condition on the environment we obtain strong ergodicity properties for the environment as seen from the position of the walker,…
In this article, we study a branching random walk in an environment which depends on the time. This time-inhomogeneous environment consists of a sequence of macroscopic time intervals, in each of which the law of reproduction remains…
This paper considers the problem of randomized influence maximization over a Markovian graph process: given a fixed set of nodes whose connectivity graph is evolving as a Markov chain, estimate the probability distribution (over this fixed…
We study ergodic properties of some Markov chains models in random environments when the random Markov kernels that define the dynamic satisfy some usual drift and small set conditions but with random coefficients. In particular, we adapt a…
Algorithms and dynamics over networks often involve randomization, and randomization may result in oscillating dynamics which fail to converge in a deterministic sense. In this paper, we observe this undesired feature in three applications,…
We consider a stochastic individual based model where each predator searches during a random time and then manipulates its prey or rests. The time distributions may be non-exponential. An age structure allows to describe these interactions…
We analyze the dynamics of a population of independent random walkers on a graph and develop a simple model of epidemic spreading. We assume that each walker visits independently the nodes of a finite ergodic graph in a discrete-time…
The random walk with hyperbolic probabilities that we are introducing is an example of stochastic diffusion in a one-dimensional heterogeneous media. Although driven by site-dependent one-step transition probabilities, the process retains…
Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition…