Related papers: Approximations for time-dependent distributions in…
We consider the class of Markovian processes defined by the equation $\dd x /\dd t = -\beta x + \sum_k z_k \delta (t-t_k)$. Such processes are encountered in systems (like coalescing systems) where dynamics creates discrete upward jumps at…
The position $x(t)$ of a particle diffusing in a one-dimensional uncorrelated and time dependent random medium is simply Gaussian distributed in the typical direction, i.e. along the ray $x=v_0 t$, where $v_0$ is the average drift. However,…
The movement of organisms and cells can be governed by occasional long distance runs, according to an approximate L\'evy walk. For T cells migrating through chronically-infected brain tissue, runs are further interrupted by long pauses, and…
We study the policy evaluation problem in multi-agent reinforcement learning, modeled by a Markov decision process. In this problem, the agents operate in a common environment under a fixed control policy, working together to discover the…
This paper is an overview of the classical level crossing problem which is studied extensively in the literature and is fundamental in many branches of applied probability. We discuss a number of approximations with an emphasis on their…
The continuous time random walk model plays an important role in modeling of so called anomalous diffusion behaviour. One of the specific property of such model are constant time periods visible in trajectory. In the continuous time random…
We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the…
This paper deals with ergodic theorems for particular time-inhomogeneous Markov processes, whose the time-inhomogeneity is asymptotically periodic. Under a Lyapunov/minorization condition, it is shown that, for any measurable bounded…
We develop a martingale approximation approach to studying the limiting behavior of quadratic forms of Markov chains. We use the technique to examine the asymptotic behavior of lag-window estimators in time series and we apply the results…
Ramaswami showed recently that standard Brownian motion arises as the limit of a family of Markov-modulated linear fluid processes. We pursue this analysis with a fluid approximation for Markov-modulated Brownian motion. Furthermore, we…
Monte Carlo (MC) simulations of transport in random porous networks indicate that for high variances of the log-normal permeability distribution, the transport of a passive tracer is non-Fickian. Here we model this non-Fickian dispersion in…
We consider a two-node queue modeled as a two-dimensional random walk. In particular, we consider the case that one or both queues have finite buffers. We develop an approximation scheme based on the Markov reward approach to error bounds…
We analyse diffusion dynamics on weakly-coupled networks (interconnected networks) by means of separation of time scales. Using an adiabatic approximation we reduced the system dynamics to a Markov chain with aggregated variables and…
Direct simulation of biomolecular dynamics in thermal equilibrium is challenging due to the metastable nature of conformation dynamics and the computational cost of molecular dynamics. Biased or enhanced sampling methods may improve the…
We investigate the hitting times of random walks on graphs, where a hitting time is defined as the number of steps required for a random walker to move from one node to another. While much of the existing literature focuses on calculating…
Stochastic Event Timing is a fundamental issue in developing both analytic and simulation models for stochastic systems. Generalized Erlang distributions are quite useful for generating those random events in a quite general way by…
Continuous Time Random Maxima (CTRM) are a generalization of classical extreme value theory: Instead of observing random events at regular intervals in time, the waiting times between the events are also random variables with arbitrary…
In this brief note, we find formulas for the distribution and the transition probability matrices of a stochastic process described as a time-reversion in a finite time window of a Markov chain, with cluster observation of the Markov state…
An approximate maximum likelihood method of estimation of diffusion parameters $(\vartheta,\sigma)$ based on discrete observations of a diffusion $X$ along fixed time-interval $[0,T]$ and Euler approximation of integrals is analyzed. We…
Scaled type Markov renewal processes generalize classical renewal processes: renewal times come from a one parameter family of probability laws and the sequence of the parameters is the trajectory of an ergodic Markov chain. Our primary…