Related papers: Constructing Strong Markov Processes
We formally extend the notion of Markov order to open quantum processes by accounting for the instruments used to probe the system of interest at different times. Our description recovers the classical Markov order property in the…
The objective of this paper is to examine the restriction of a right process on a Radon topological space, excluding a negligible set, and investigate whether the restricted object can induce a Markov process with desirable properties. We…
Macdonald processes are certain probability measures on two-dimensional arrays of interlacing particles introduced by Borodin and Corwin (arXiv:1111.4408 [math.PR]). They are defined in terms of nonnegative specializations of the Macdonald…
General-purpose, intelligent, learning agents cycle through sequences of observations, actions, and rewards that are complex, uncertain, unknown, and non-Markovian. On the other hand, reinforcement learning is well-developed for small…
Discrete latent space models have recently achieved performance on par with their continuous counterparts in deep variational inference. While they still face various implementation challenges, these models offer the opportunity for a…
We study a Markov decision problem in which the state space is the set of finite marked point configurations in the plane, the actions represent thinnings, the reward is proportional to the mark sum which is discounted over time, and the…
Consider a parametric statistical model $P(\mathrm{d}x|\theta)$ and an improper prior distribution $\nu(\mathrm{d}\theta)$ that together yield a (proper) formal posterior distribution $Q(\mathrm{d}\theta|x)$. The prior is called strongly…
We study the mechanical unfolding of a simple model protein. The Langevin dynamics results are analyzed using Markov-model methods which allow to describe completely the configurational space of the system. Using transition path theory we…
Recent research in decision theoretic planning has focussed on making the solution of Markov decision processes (MDPs) more feasible. We develop a family of algorithms for structured reachability analysis of MDPs that are suitable when an…
A construction is given of Markov partitions for some rational maps, which persist over regions of parameter space, not confined to single hyperbolic components. The set on which the Markov partition exists, and its boundary, are analysed.
We study the large deviations principle for one dimensional, continuous, homogeneous, strong Markov processes that do not necessarily behave locally as a Wiener process. Any strong Markov process $X_{t}$ in $\mathbb{R}$ that is continuous…
A sequence of real numbers (x_n) is Benford if the significands, i.e. the fraction parts in the floating-point representation of (x_n) are distributed logarithmically. Similarly, a discrete-time irreducible and aperiodic finite-state Markov…
We study properties of strongly separately continuous mappings defined on subsets of products of topological spaces equipped with the topology of pointwise convergence. In particular, we give a necessary and sufficient condition for a…
Suppose that $(X_t)_{t \ge 0}$ is a one-dimensional Brownian motion with negative drift $-\mu$. It is possible to make sense of conditioning this process to be in the state $0$ at an independent exponential random time and if we kill the…
We investigate the long-time behavior of solutions to a stochastically forced one-dimensional Navier-Stokes system, describing the motion of a compressible viscous fluid, in the case of linear pressure law. We prove existence of an…
We consider Markov processes of cubic stochastic (in a fixed sense) matrices which are also called quadratic stochastic process (QSPs). A QSP is a particular case of a continuous-time dynamical system whose states are stochastic cubic…
The goal of this work is to formally abstract a Markov process evolving in discrete time over a general state space as a finite-state Markov chain, with the objective of precisely approximating its state probability distribution in time,…
We study the existence of densities for distributions of piecewise deterministic Markov processes. We also obtain relationships between invariant densities of the continuous time process and that of the process observed at jump times. In…
In this paper we present elementary computations for some Markov modulated counting processes, also called counting processes with regime switching. Regime switching has become an increasingly popular concept in many branches of science. In…
The classical embeddability problem asks whether a given stochastic matrix $T$, describing transition probabilities of a $d$-level system, can arise from the underlying homogeneous continuous-time Markov process. Here, we investigate the…