Related papers: Reverting Processes
A continuous-time Markov process $X$ can be conditioned to be in a given state at a fixed time $T > 0$ using Doob's $h$-transform. This transform requires the typically intractable transition density of $X$. The effect of the $h$-transform…
Consider the continuous-time Markov Branching Process. In critical case we consider a situation when the generating function of intensity of transformation of particles has the infinite second moment, but its tail regularly varies in sense…
A second-order random walk on a graph or network is a random walk where transition probabilities depend not only on the present node but also on the previous one. A notable example is the non-backtracking random walk, where the walker is…
In this brief note, we find formulas for the distribution and the transition probability matrices of a stochastic process described as a time-reversion in a finite time window of a Markov chain, with cluster observation of the Markov state…
In this paper we introduce the concept of random time changes in dynamical systems. The subordination principle may be applied to study the long time behavior of the random time systems. We show, under certain assumptions on the class of…
We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
Suppose $X$ is a Markov process on the real line (or some interval). Do the distributions of its first passage times downwards (fptd) determine its law? In this paper we treat some special cases of this question. We prove that if the fptd…
We study Markov processes conditioned so that their local time must grow slower than a prescribed function. Building upon recent work on Brownian motion with constrained local time in [5] and [33], we study transience and recurrence for a…
The use of coordinate processes for the modelling of impulse control for general Markov processes typically involves the construction of a probability measure on a countable product of copies of the path space. In addition, admissibility of…
Random walks are used for modeling various dynamics in, for example, physical, biological, and social contexts. Furthermore, their characteristics provide us with useful information on the phase transition and critical phenomena of even…
In this paper we revise the theory of turnpikes in discounted Markov decision processes, prove the turnpike theorem for the undiscounted model and apply the results to the specific random walk.
The reciprocal class of a Markov path measure is the set of all mixtures of its bridges. We give characterizations of the reciprocal class of a continuous-time Markov random walk on a graph. Our main result is in terms of some reciprocal…
In the past decade, the use of ordinal patterns in the analysis of time series and dynamical systems has become an important and rich tool. Ordinal patterns (otherwise known as a permutation patterns) are found in time series by taking $n$…
Weconsider Markov decision processes arising from a Markov model of an underlying natural phenomenon. Such phenomena are usually periodic (e.g. annual) in time, and so the Markov processes modelling them must be time-inhomogeneous, with…
Graphical Markov models combine conditional independence constraints with graphical representations of stepwise data generating processes.The models started to be formulated about 40 years ago and vigorous development is ongoing.…
We consider a class of tempered subordinators, namely a class of subordinators with one-dimensional marginal tempered distributions which belong to a family studied in [3]. The main contribution in this paper is a non-central moderate…
Datasets involving sequences of different types of events without meaningful time stamps are prevalent in many applications, for instance when extracted from textual corpora. We propose a family of models for such event sequences -- summary…
The paper studies an improved estimate for the rate of convergence for nonlinear homogeneous discrete-time Markov chains. These processes are nonlinear in terms of the distribution law. Hence, the transition kernels are dependent on the…
We analyze the properties of degree-preserving Markov chains based on elementary edge switchings in undirected and directed graphs. We give exact yet simple formulas for the mobility of a graph (the number of possible moves) in terms of its…