Related papers: Non-Markovian chains with long-range dependence an…
We consider periodic Markov chains with absorption. Applying to iterates of this periodic Markov chain criteria for the exponential convergence of conditional distributions of aperiodic absorbed Markov chains, we obtain exponential…
Markov decision processes are typically used for sequential decision making under uncertainty. For many aspects however, ranging from constrained or safe specifications to various kinds of temporal (non-Markovian) dependencies in task and…
We consider Markov processes, which describe e.g. queueing network processes, in a random environment which influences the network by determining random breakdown of nodes, and the necessity of repair thereafter. Starting from an explicit…
We discuss the response of continuous time random walks to an oscillating external field within the generalized master equation approach. We concentrate on the time dependence of the two first moments of the walker's displacements. We show…
A discrete time branching process where the offspring distribution is generation-dependent, and the number of reproductive individuals is controlled by a random mechanism is considered. This model is a Markov chain but, in general, the…
Switching ARMA models greatly enhance the standard linear models to the extent that different ARMA model is allowed in a different regime, and the regime switching is typically assumed a Markov chain on the finite states of potential…
We address the problem of community detection in networks by introducing a general definition of Markov stability, based on the difference between the probability fluxes of a Markov chain on the network at different time scales. The…
In this paper we study the long term evolution of a continuous time Markov chain formed by two interacting birth-and-death processes. The interaction between the processes is modelled by transition rates which are functions with suitable…
In this paper, we study Markov-dependent reflected autoregressive processes, and other related models the analysis of which results in a vector-valued fixed-point functional equation of a certain type. In queueing terms, such processes…
Localized perturbations in a real-world network have the potential to trigger cascade failures at the whole system level, hindering its operations and functions. Standard approaches analytically tackling this problem are mostly based either…
In this paper, we present new Poisson-type deviation inequalities for continuous-time Markov chains whose Wasserstein curvature or $\Gamma$-curvature is bounded below. Although these two curvatures are equivalent for Brownian motion on…
In this research paper, weighted / unweighted, directed / undirected graphs are associated with interesting Discrete Time Markov Chains (DTMCs) as well as Continuous Time Markov Chains (CTMCs). The equilibrium / transient behaviour of such…
We introduce and study some queueing models with random resetting, including Markovian and non--Markovian models. The Markovian models include M/M/$\infty$, M/M/r and M/M/1+M queues with random resetting, in which a continuous-time Markov…
The ensemble properties and time-averaged observables of a memory-induced diffusive-superdiffusive transition are studied. The model consists in a random walker whose transitions in a given direction depend on a weighted linear combination…
Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically.…
A switching random walk, commonly known under the misnomer `oscillating random walk', is a real-valued Markov chain whose distribution of increments is determined by the sign of the current position. We explicitly identify an invariant…
A theory of systems with long-range correlations based on the consideration of binary N-step Markov chains is developed. In our model, the conditional probability that the i-th symbol in the chain equals zero (or unity) is a linear function…
Consider a Markov chain with finite state space and suppose you wish to change time replacing the integer step index $n$ with a random counting process $N(t)$. What happens to the mixing time of the Markov chain? We present a partial reply…
Multi-state models are frequently applied for representing processes evolving through a discrete set of state. Important classes of multi-state models arise when transitions between states may depend on the time since entry into the current…
We consider a stable open queuing network as a steady non-equilibrium system of interacting particles. The network is completely specified by its underlying graphical structure, type of interaction at each node, and the Markovian transition…