Related papers: Conditioned real self-similar Markov processes
The quantitative long time behavior of absorbing, finite, irreducible Markov processes is considered. Via Doob transforms, it is shown that only the knowledge of the ratio of the values of the underlying first Dirichlet eigenvector is…
The steady states of dynamical processes can exhibit stable nontrivial phases, which can also serve as fault-tolerant classical or quantum memories. For Markovian quantum (classical) dynamics, these steady states are extremal eigenvectors…
We consider a general honest homogeneous continuous-time Markov process with restarts. The process is forced to restart from a given distribution at time moments generated by an independent Poisson process. The motivation to study such…
The Markov assumption in Markov Decision Processes (MDPs) is fundamental in reinforcement learning, influencing both theoretical research and practical applications. Existing methods that rely on the Bellman equation benefit tremendously…
When statistical analyses consider multiple data sources, Markov melding provides a method for combining the source-specific Bayesian models. Markov melding joins together submodels that have a common quantity. One challenge is that the…
We characterize the value function and the optimal stopping time for a large class of optimal stopping problems where the underlying process to be stopped is a fairly general Markov process. The main result is inspired by recent findings…
In this paper, we develop necessary and sufficient conditions for the validity of a martingale approximation for the partial sums of a stationary process in terms of the maximum of consecutive errors. Such an approximation is useful for…
We are interested in understanding stability (almost sure boundedness) of stochastic approximation algorithms (SAs) driven by a `controlled Markov' process. Analyzing this class of algorithms is important, since many reinforcement learning…
We study the long time behaviour of a Markov process evolving in $\mathbb{N}$ and conditioned not to hit 0. Assuming that the process comes back quickly from infinity, we prove that the process admits a unique quasi-stationary distribution…
In this paper, we obtain a Lamperti type representation for real-valued self-similar Markov processes, killed at their hitting time of zero. Namely, we represent real-valued self-similar Markov processes as time changed multiplicative…
We develop criteria for recurrence and transience of one-dimensional Markov processes which have jumps and oscillate between $+\infty$ and $-\infty$. The conditions are based on a Markov chain which only consists of jumps (overshoots) of…
A mathematical framework for Continuous Time Finance based on operator algebraic methods offers a new direct and entirely constructive perspective on the field and leads to new numerical analysis techniques. This is partly a review paper as…
We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this…
A general affine Markov semigroup is formulated as the convolution of a homogeneous one with a skew convolution semigroup. We provide some sufficient conditions for the regularities of the homogeneous affine semigroup and the skew…
An $\mathbb{R}^d$-valued Markov process $X^{(x)}_t=(X^{1,x_1}_t,\dots,X^{d,x_d}_t)$, $t\ge0,x\in\mathbb{R}^d$ is said to be multi-self-similar with index $(\alpha_1,\dots,\alpha_d)\in[0,\infty)^d$ if the identity in law…
In this paper we consider the convergence of the conditional entropy to the entropy rate for Markov chains. Convergence of certain statistics of long range dependent processes, such as the sample mean, is slow. It has been shown in Carpio…
A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…
Previous authors have considered optimal stopping problems driven by the running maximum of a spectrally negative L\'evy process $X$, as well as of a one-dimensional diffusion. Many of the aforementioned results are either implicitly or…
We study the asymptotic behavior of Markov operators $P_\mu$ defined by convolution with a probability measure $\mu$ on the unit circle $\mathbb T$. We prove that when $\mu$ is adapted, $P_\mu$ satisfies Doeblin's condition if and only if…
Positive self-similar Markov processes (pssMp) are positive Markov processes that satisfy the scaling property and it is known that they can be represented as the exponential of a time-changed L\'evy process via Lamperti representation. In…