Related papers: On Deterministic Markov Processes: Expandability a…
Using the age-structure formalism, we definitely establish connections between semi-Markov processes and the dynamics of open quantum systems that satisfy the Markov quantum master equations. A generalized Feynman-Kac formula of the…
We address a class of Markov jump linear systems that are characterized by the underlying Markov process being time-inhomogeneous with a priori unknown transition probabilities. Necessary and sufficient conditions for uniform stochastic…
Two different Markov jump processes driven out of equilibrium by constant thermodynamic forces may have identical current fluctuations in the stationary state. The concept of dynamical equivalence classes emerges from this statement as…
The paper deals with a family of jump Markov process defined in a medium with a periodic or locally periodic microstructure. We assume that the generator of the process is a zero order convolution type operator with rapidly oscillating…
We study a one-dimensional random walk with memory in which the step lengths to the left and to the right evolve at each step in order to reduce the wandering of the walker. The feedback is quite efficient and lead to a non-diffusive walk.…
The martingale comparison method is extended to derive comparison results for path-independent functions for general semimartingales. Our approach allows to dismiss with the Markovian assumption on one of the processes made in previous…
Stochastic exclusion processes play an integral role in the physics of non-equilibrium statistical mechanics. These models are Markovian processes, described by a classical master equation. In this paper a quantum mechanical version of a…
Markov decision processes (MDPs) are a standard model for sequential decision-making problems and are widely used across many scientific areas, including formal methods and artificial intelligence (AI). MDPs do, however, come with the…
Let $E$ be a locally compact separable metric space and $m$ be a positive Radon measure on it. Given a nonnegative function $k$ defined on $E\times E$ off the diagonal whose anti-symmetric part is assumed to be less singular than the…
We study a $d$-dimensional random walk with exponentially distributed increments conditioned so that the components stay ordered (in the sense of Doob). We find explicitly a positive harmonic function $h$ for the killed process and then…
In the development of stochastic integration and the theory of semimartingales, Markov processes have been a constant source of inspiration. Despite this historical interweaving, it turned out that semimartingales should be considered the…
A labelled Markov decision process is a labelled Markov chain with nondeterminism, i.e., together with a strategy a labelled MDP induces a labelled Markov chain. The model is related to interval Markov chains. Motivated by applications of…
Dynamical processes can be classified in various ways as deterministic or stochastic, and continuous or discrete time. All these types can be studied by the path-spaces they generate, and stationary measures on that path-space. Such…
We present a general framework for applying learning algorithms and heuristical guidance to the verification of Markov decision processes (MDPs). The primary goal of our techniques is to improve performance by avoiding an exhaustive…
A general theory is developed to study individual based models which are discrete in time. We begin by constructing a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are…
We present recent results on Piecewise Deterministic Markov Processes (PDMPs), involved in biological modeling. PDMPs, first introduced in the probabilistic literature by Davis (1984), are a very general class of Markov processes and are…
We consider importance sampling as well as other properly weighted samples with respect to a target distribution $\pi$ from a different point of view. By considering the associated weights as sojourn times until the next jump, we define…
Piecewise Deterministic Markov Processes (PDMPs) such as the Bouncy Particle Sampler and the Zig-Zag Sampler, have gained attention as continuous-time counterparts of classical Markov chain Monte Carlo. We study their transient regime under…
We consider the moving particle process in Rd which is defined in the following way. There are two independent sequences (Tk) and (dk) of random variables. The variables Tk are non negative and form an increasing sequence, while variables…
We construct a large class of non-Markovian master equations that describe the dynamics of open quantum systems featuring strong memory effects, which relies on a quantum generalization of the concept of classical semi-Markov processes.…