Related papers: Delayed logistic equation as a limit of long memor…
We describe a generalization of the Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) which is able to encode prior information that state transitions are more likely between "nearby" states. This is accomplished by defining a…
For a generalized step reinforced random walk, starting from the origin, the first step is taken according to the first element of an innovation sequence. Then in subsequent epochs, it recalls a past epoch with probability proportional to a…
A fundamental problem of non-equilibrium statistical mechanics is the derivation of macroscopic transport equations in the hydrodynamic limit. The rigorous study of such limits requires detailed information about rates of convergence to…
We carry out a delay stability analysis (i.e., determine conditions under which expected steady-state delays at a queue are finite) for a simple 3-queue system operated under the Max-Weight scheduling policy, for the case where one of the…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
In this article, we consider a Markov process X, starting from x and solving a stochastic differential equation, which is driven by a Brownian motion and an independent pure jump component exhibiting state-dependent jump intensity and…
The solution to Poisson's equation arise in many Markov chain and Markov jump process settings, including that of the central limit theorem, value functions for average reward Markov decision processes, and within the gradient formula for…
In this short paper, we consider discrete-time Markov chains on lattices as approximations to continuous-time diffusion processes. The approximations can be interpreted as finite difference schemes for the generator of the process. We…
We investigate the mixing properties of a finite Markov chain in random environment defined as a mixture of a deterministic chain and a chain whose state space has been permuted uniformly at random. This work is the counterpart of a…
We consider a queueing system with $n$ parallel queues operating according to the so-called "supermarket model" in which arriving customers join the shortest of $d$ randomly selected queues. Assuming rate $n\lambda_{n}$ Poisson arrivals and…
Many probabilistic inference problems such as stochastic filtering or the computation of rare event probabilities require model analysis under initial and terminal constraints. We propose a solution to this bridging problem for the widely…
Recurrent Neural Networks (RNNs) are widely recognized for their proficiency in modeling temporal dependencies, making them highly prevalent in sequential data processing applications. Nevertheless, vanilla RNNs are confronted with the…
We consider $N$ counters taking integer values which are subject to the following dynamics. At every time, a pair of distinct counters is chosen uniformly at random and their states are updated according to the following rule. If the states…
We consider Markov processes in continuous time with state space $\posint^N$ and provide two sufficient conditions and one necessary condition for the existence of moments $E(\|X(t)\|^r)$ of all orders $r \in \nat$ for all $t \geq 0$. The…
We study a one-dimensional Markov modulated random walk with jumps. It is assumed that amplitudes of jumps as well as a chosen velocity regime are random and depend on a time spent by the process at a previous state of the underlying Markov…
Let $N$ and $M$ be positive integers satisfying $1\le M\le N$, and let $0<p_0<p_1<1$. Define a process $\{X_n\}_{n=0}^\infty$ on $\mathbb{Z}$ as follows. At each step, the process jumps either one step to the right or one step to the left,…
Motivated by applications in large-scale and multi-agent reinforcement learning, we study the non-asymptotic performance of stochastic approximation (SA) schemes with delayed updates under Markovian sampling. While the effect of delays has…
We examine the non-Markovian nature of human mobility by exposing the inability of Markov models to capture criticality in human mobility. In particular, the assumed Markovian nature of mobility was used to establish a theoretical upper…
In this paper, we study joint queue-aware and channel-aware scheduling of arbitrarily bursty traffic over multi-state time-varying channels, where the bursty packet arrival in the network layer, the backlogged queue in the data link layer,…
We are interested in the asymptotic behavior of Markov chains on the set of positive integers for which, loosely speaking, large jumps are rare and occur at a rate that behaves like a negative power of the current state, and such that small…