Related papers: Constructing Strong Markov Processes
Let $R$ be a continuous-time Markov process on the time interval $[0,1]$ with values in some state space $X$. We transform this reference process $R$ into $P:=f(X_0)\exp (-\int_0^1 V_t(X_t) dt) g(X_1)\,R$ where $f,g$ are nonnegative…
Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…
This article provides an introductory tutorial on structural results in partially observed Markov decision processes (POMDPs). Typically, computing the optimal policy of a POMDP is computationally intractable. We use lattice program- ming…
Semi-Markov processes represent a well known and widely used class of random processes in classical probability theory. Here, we develop an extension of this type of non-Markovian dynamics to the quantum regime. This extension is…
We study finite state random dynamical systems (RDS) and their induced Markov chains (MC) as stochastic models for complex dynamics. The linear representation of deterministic maps in RDS are matrix-valued random variables whose…
In this paper, we discuss hypercontractivity for the Markov semigroup $P_t$ which is generated by segment processes associated with a range of functional SDEs of neutral type. As applications, we also reveal that the semigroup $P_t$…
It is shown by constructing Rohlins canonical measures that for a strictly stationary, d-dimensional vector-valued process X there exists another strictly stationary d-dimensional process U with uniform one-dimensional marginals and with…
Extensions of Kemeny's constant, as derived for irreducible finite Markov chains in discrete time, to Markov renewal processes and Markov chains in continuous time are discussed. Three alternative Kemeny's functions and their variants are…
We study a Markov random process describing a muscle molecular motor behavior. Every motor is either bound up with a thin filament or unbound. In the bound state the motor creates a force proportional to its displacement from the neutral…
We construct a large class of non-Markovian master equations that describe the dynamics of open quantum systems featuring strong memory effects, which relies on a quantum generalization of the concept of classical semi-Markov processes.…
Discrete time random dynamical systems with countably many maps which admit countable Markov partitions on complete metric spaces such that the resulting Markov systems are uniform continuous and contractive are considered. A notion of a…
Noncolliding Brownian motion (Dyson's Brownian motion model with parameter $\beta=2$) and noncolliding Bessel processes are determinantal processes; that is, their space-time correlation functions are represented by determinants. Under a…
This paper studies the exponential stability of random matrix products driven by a general (possibly unbounded) state space Markov chain. It is a cornerstone in the analysis of stochastic algorithms in machine learning (e.g. for parameter…
An $\mathbb{R}^d$-valued Markov process $X^{(x)}_t=(X^{1,x_1}_t,\dots,X^{d,x_d}_t)$, $t\ge0,x\in\mathbb{R}^d$ is said to be multi-self-similar with index $(\alpha_1,\dots,\alpha_d)\in[0,\infty)^d$ if the identity in law…
Consider a stochastic process $\{X(t)\}$ on a finite state space $ {\sf X}=\{1,\dots, d\}$. It is conditionally Markov, given a real-valued `input process' $\{\zeta(t)\}$. This is assumed to be small, which is modeled through the scaling,…
Constrained Markov processes, such as reflecting diffusions, behave as an unconstrained process in the interior of a domain but upon reaching the boundary are controlled in some way so that they do not leave the closure of the domain. In…
We consider a finite state discrete time process X. Without loss of generality the finite state space can be identified with the set of unit vectors {e1, e2, . . . , eN} with ei = (0, . . . , 0, 1, 0, . . . , 0)0 2 RN. For a Markov chain…
We consider infinite weighted graphs $G$, i.e., sets of vertices $V$, and edges $E$ assumed countable infinite. An assignment of weights is a positive symmetric function $c$ on $E$ (the edge-set), conductance. From this, one naturally…
The time at which a one-dimensional continuous strong Markov process attains a boundary point of its state space is a discontinuous path functional and it is, therefore, unclear whether the exit time can be approximated by hitting times of…
We start by considering infinite dimensional Markovian dynamics in R^m generated by operators of hypocoercive type and for such models we obtain short and long time pointwise estimates for all the derivatives, of any order and in any…