Related papers: Exit Times for a Discrete Markov Additive Process
We build on a previous statistical model for distributed systems and formulate it in a way that the deterministic and stochastic processes within the system are clearly separable. We show how internal fluctuations can be analysed in a…
The density-dependent Markov chain (DDMC) introduced in \cite{Kurtz1978} is a continuous time Markov process applied in fields such as epidemics, chemical reactions and so on. In this paper, we give moderate deviation principles of paths of…
At high levels, the asymptotic distribution of a stationary, regularly varying Markov chain is conveniently given by its tail process. The latter takes the form of a geometric random walk, the increment distribution depending on the sign of…
We consider the problem of estimating the transition rate matrix of a continuous-time Markov chain from a finite-duration realisation of this process. We approach this problem in an imprecise probabilistic framework, using a set of prior…
We study Exo-MDPs, a structured class of Markov Decision Processes (MDPs) where the state space is partitioned into exogenous and endogenous components. Exogenous states evolve stochastically, independent of the agent's actions, while…
Continuous-time Markov chains are used to model stochastic systems where transitions can occur at irregular times, e.g., birth-death processes, chemical reaction networks, population dynamics, and gene regulatory networks. We develop a…
Let $(X_t)_{t = 0 }^{\infty}$ be an irreducible reversible discrete time Markov chain on a finite state space $\Omega $. Denote its transition matrix by $P$. To avoid periodicity issues (and thus ensuring convergence to equilibrium) one…
Most previous contributions to BSDEs, and the related theories of nonlinear expectation and dynamic risk measures, have been in the framework of continuous time diffusions or jump diffusions. Using solutions of BSDEs on spaces related to…
We consider trap models on Z^d, namely continuous time Markov jump process on Z^d with embedded chain given by a generic discrete time random walk, and whose mean waiting time at x is given by tau_x, with tau = (tau_x, x in Z^d) a family of…
Spatial birth-and-death processes with time dependent rates are obtained as solutions to certain stochastic equations. The existence, uniqueness, uniqueness in law and the strong Markov property of unique solutions are proven when the…
We consider the discrete-time migration-recombination equation, a deterministic, nonlinear dynamical system that describes the evolution of the genetic type distribution of a population evolving under migration and recombination in a law of…
As a main step in the numerical solution of control problems in continuous time, the controlled process is approximated by sequences of controlled Markov chains, thus discretising time and space. A new feature in this context is to allow…
Cumulative prospect theory (CPT) is the first theory for decision-making under uncertainty that combines full theoretical soundness and empirically realistic features [P.P. Wakker - Prospect theory: For risk and ambiguity, Page 2]. While…
A Markov Additive Process is a bi-variate Markov process $(\xi,J)=\big((\xi_t,J_t),t\geq0\big)$ which should be thought of as a multi-type L\'evy process: the second component $J$ is a Markov chain on a finite space $\{1,\ldots,K\}$, and…
We derive some key extremal features for $k$th order Markov chains that can be used to understand how the process moves between an extreme state and the body of the process. The chains are studied given that there is an exceedance of a…
We make use of matrix representations of completely positive maps in order to study open quantum dynamics on graphs, with emphasis on quantum walks and the associated trajectories obtained via a monitoring of the position. We discuss the…
In this paper, we perform a more general analysis on the discrete effects of some boundary schemes of the popular one- to three-dimensional DnQq multiple-relaxation-time lattice Boltzmann model for convection-diffusion equation (CDE).…
Developments in dynamical systems theory provides new support for the macroscale modelling of pdes and other microscale systems such as Lattice Boltzmann, Monte Carlo or Molecular Dynamics simulators. By systematically resolving subgrid…
The motivation of this work is to extend the techniques of higher order random walks on simplicial complexes to analyze mixing times of Markov chains for combinatorial problems. Our main result is a sharp upper bound on the second…
In this note we consider a Markov chain formed by a finite system of interacting birth-and-death processes on a finite state space. We study an asymptotic behaviour of the Markov chain as its state space becomes large. In particular, we…