Related papers: Rank-driven Markov processes
Reinforcement learning algorithms are typically designed for generic Markov Decision Processes (MDPs), where any state-action pair can lead to an arbitrary transition distribution. In many practical systems, however, only a subset of the…
The interplay between topology and dynamics in complex networks is a fundamental but widely unexplored problem. Here, we study this phenomenon on a prototype model in which the network is shaped by a dynamical variable. We couple the…
We build on a previous statistical model for distributed systems and formulate it in a way that the deterministic and stochastic processes within the system are clearly separable. We show how internal fluctuations can be analysed in a…
Motivated by the study of the time evolution of random dynamical systems arising in a vast variety of domains --- ranging from physics to ecology ---, we establish conditions for the occurrence of a non-trivial asymptotic behaviour for…
The short-time and long-time dynamics of the Bak-Sneppen model of biological evolution are investigated using the damage spreading technique. By defining a proper Hamming distance measure, we are able to make it exhibits an initial…
The molecular evolution in a gene regulatory network is classically modeled by Markov jump processes. However, the direct simulation of such models is extremely time consuming. Indeed, even the simplest Markovian model, such as the…
In this paper, we provide a novel algorithm for solving planning and learning problems of Markov decision processes. The proposed algorithm follows a policy iteration-type update by using a rank-one approximation of the transition…
Complex systems made of interacting elements are commonly abstracted as networks, in which nodes are associated with dynamic state variables, whose evolution is driven by interactions mediated by the edges. Markov processes have been the…
We study K-processes, which are Markov processes in a denumerable state space, all of whose elements are stable, with the exception of a single state, starting from which the process enters finite sets of stable states with uniform…
We study the simple evolutionary process in which we repeatedly find the least fit agent in a population of agents and give it a new fitness which is chosen independently at random from a specified distribution. We show that many of the…
We consider a class of stochastic dynamical systems, called piecewise deterministic Markov processes, with states $(x, \s)\in \O\times \G$, $\O$ being a region in $\bbR^d$ or the $d$--dimensional torus, $\G$ being a finite set. The…
The Bak-Sneppen model of self-organized biological evolution of an infinite ecosystem of randomly interacting species is represented in terms of an infinite set of variables which can be considered as an analog to the set of integrals of…
We develop a class of exponential-family point processes based on a latent social space to model the coevolution of social structure and behavior over time. Temporal dynamics are modeled as a discrete Markov process specified through…
We consider an elementary model for self-organised criticality, the activated random walk on the complete graph. We introduce a discrete time Markov chain as follows. At each time step, we add an active particle at a random vertex and let…
We study very simple sorting algorithms based on a probabilistic comparator model. In our model, errors in comparing two elements are due to (1) the energy or effort put in the comparison and (2) the difference between the compared…
It is possible to represent each of a number of Markov chains as an evolving sequence of connected subsets of a directed acyclic graph that grow in the following way: initially, all vertices of the graph are unoccupied, particles are fed in…
We consider a large family of branching-selection particle systems. The branching rate of each particle depends on its rank and is given by a function $b$ defined on the unit interval. There is also a killing measure $D$ supported on the…
We derive some key extremal features for $k$th order Markov chains that can be used to understand how the process moves between an extreme state and the body of the process. The chains are studied given that there is an exceedance of a…
We consider a Markovian evolution on point processes, the $\Psi$--process, on the unit interval in which points are added according to a rule that depends only on the spacings of the existing point configuration. Having chosen a spacing, a…
Consider a continuous time particle system $\eta^t=(\eta^t(k),k\in \mathbb{L})$, indexed by a lattice $\mathbb{L}$ which will be either $\mathbb{Z}$, $\mathbb{Z}/n\mathbb{Z}$, a segment $\{1,\cdots, n\}$, or $\mathbb{Z}^d$, and taking its…