Related papers: Delayed logistic equation as a limit of long memor…

We consider the problem of steering a system with unknown, stochastic dynamics to satisfy a rich, temporally layered task given as a signal temporal logic formula. We represent the system as a Markov decision process in which the states are…

Many systems in biology, physics and chemistry can be modeled through ordinary differential equations, which are piecewise smooth, but switch between different states according to a Markov jump process. In the fast switching limit, the…

Under unitary evolution, systems move gradually from state to state. An unstable atom has amplitude in its original state after many lifetimes ($\tau_L$). But in the laboratory, transitions seem to go instantaneously, as suggested by the…

This paper deals a continuous-time state-dependent jump linear system, a particular kind of stochastic switching system. In particular, we consider a situation when the transition rate of the random jump process depends on the state…

Classically, solution theories for state-dependent delay equations are developed in spaces of continuous or continuously differentiable functions. The former can be technically challenging to apply in as much as suitably Lipschitz…

This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying…

We consider two types of discrete-time Markov chains where the state space is a graded poset and the transitions are taken along the covering relations in the poset. The first type of Markov chain goes only in one direction, either up or…

In this paper we consider the problem of parameter inference for Markov jump process (MJP) representations of stochastic kinetic models. Since transition probabilities are intractable for most processes of interest yet forward simulation is…

This paper deals with the long run average continuous control problem of piecewise deterministic Markov processes (PDMP's) taking values in a general Borel space and with compact action space depending on the state variable. The control…

Let {X_n,n\geq0} be a Markov chain on a general state space X with transition probability P and stationary probability \pi. Suppose an additive component S_n takes values in the real line R and is adjoined to the chain such that…

We consider irreducible reversible discrete time Markov chains on a finite state space. Mixing times and hitting times are fundamental parameters of the chain. We relate them by showing that the mixing time of the lazy chain is equivalent…

A $p$-jump process is a piecewise deterministic Markov process with jumps by a factor of $p$. We prove a limit theorem for such processes on the unit interval. Via duality with respect to probability generating functions, we deduce limiting…

Random walks are a fundamental model in applied mathematics and are a common example of a Markov chain. The limiting stationary distribution of the Markov chain represents the fraction of the time spent in each state during the stochastic…

Let $X$ be the constrained random walk on ${\mathbb Z}_+^2$ taking the steps $(1,0)$, $(-1,1)$ and $(0,-1)$ with probabilities $\lambda < (\mu_1\neq \mu_2)$; in particular, $X$ is assumed stable. Let $\tau_n$ be the first time $X$ hits…

In this paper, we present a novel approach to modeling long-term dependencies in sequential data by introducing a gated recurrent unit (GRU) with a weighted time-delay feedback mechanism. Our proposed model, named $\tau$-GRU, is a…

Negative differential mobility is the phenomenon in which the velocity of a particle decreases when the force driving it increases. We study this phenomenon in Markov jump models where a particle moves in the presence of walls that act as…

In recent years it has become clear that the brain maintains a temporal memory of recent events stretching far into the past. This paper presents a neurally-inspired algorithm to use a scale-invariant temporal representation of the past to…

Recently several CSMA algorithms based on the Glauber dynamics model have been proposed for multihop wireless scheduling, as viable solutions to achieve the throughput optimality, yet are simple to implement. However, their delay…

We present the linear-stability analysis of synchronised states in coupled time-delay systems. There exists a synchronisation threshold, for which we derive upper bounds, which does not depend on the delay time. We prove that at least for…

We set out a general procedure which allows the approximation of certain Markov chains by the solutions of differential equations. The chains considered have some components which oscillate rapidly and randomly, while others are close to…