Related papers: Some simple but challenging Markov processes
Markov models are widely used to describe processes of stochastic dynamics. Here, we show that Markov models are a natural consequence of the dynamical principle of Maximum Caliber. First, we show that when there are different possible…
Markov-modulated Brownian motion is a popular tool to model continuous-time phenomena in a stochastic context. The main quantity of interest is the invariant density, which satisfies a differential equation associated with the quadratic…
This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of…
Both stochastic and PDE modeling approaches have been used and compared in various context in biology. Typically, stochastic models are easier to parameterize, can be used to integrate underlying biological phenomena, but hard to analyze…
A hidden Markov process is a well known concept in information theory and is used for a vast range of applications such as speech recognition and error correction. We bridge between two disciplines, experimental physics and advanced…
Rule-based modelling allows to represent molecular interactions in a compact and natural way. The underlying molecular dynamics, by the laws of stochastic chemical kinetics, behaves as a continuous-time Markov chain. However, this Markov…
Piecewise deterministic Markov processes (PDMPs) are a class of continuous-time Markov processes that were recently used to develop a new class of Markov chain Monte Carlo algorithms. However, the implementation of the processes is…
We present an algorithm that can efficiently compute a broad class of inferences for discrete-time imprecise Markov chains, a generalised type of Markov chains that allows one to take into account partially specified probabilities and other…
In this paper, we propose a novel class of Piecewise Deterministic Markov Processes (PDMPs) that are designed to sample from probability distributions $\pi$ supported on a convex set $\mathcal{M}$. This class of PDMPs adapts the concept of…
Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…
We focus on a class of BSDEs driven by a cadlag martingale and corresponding Markov type BSDE which arise when the randomness of the driver appears through a Markov process. To those BSDEs we associate a deterministic problem which, when…
Many psychological experiments have subjects repeat a task to gain the statistical precision required to test quantitative theories of psychological performance. In such experiments, time-on-task can have sizable effects on performance,…
We study invariant boundary conditions for one dimensional discrete Gaussian Markov processes, basic toy models of spatial Markov processes in statistical mechanics. More precisely, we give a decomposition of boundary objects in a non…
At the scale of the individual cell, protein production is a stochastic process with multiple time scales, combining quick and slow random steps with discontinuous and smooth variation. Hybrid stochastic processes, in particular…
Most exact algorithms for general partially observable Markov decision processes (POMDPs) use a form of dynamic programming in which a piecewise-linear and convex representation of one value function is transformed into another. We examine…
Finite stochastic Markov models play a major role for modelling biochemical pathways. Such models are a coarse-grained description of the underlying microscopic dynamics and can be considered mesoscopic. The level of coarse-graining is to a…
A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be…
The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant "additivity" properties: (i) existence of a…
Markov combination is an operation that takes two statistical models and produces a third whose marginal distributions include those of the original models. Building upon and extending existing work in the Gaussian case, we develop Markov…
When performing Bayesian computations in practice, one is often faced with the challenge that the constituent model components and/or the data are only available in a distributed fashion, e.g. due to privacy concerns or sheer volume. While…