Related papers: A Compositional Framework for Markov Processes
We have shown recently that a Markov process conditioned on rare events involving time-integrated random variables can be described in the long-time limit by an effective Markov process, called the driven process, which is given…
We consider a continuous-time financial market with an asset whose price is modeled by a linear stochastic differential equation with drift and volatility switching driven by a uniformly ergodic jump Markov process with a countable state…
We consider a Markovian approximation, of weak coupling type, to an open system perturbation involving emission, absorption and scattering by reservoir quanta. The result is the general form for a quantum stochastic flow driven by creation,…
We describe an approach for exploiting structure in Markov Decision Processes with continuous state variables. At each step of the dynamic programming, the state space is dynamically partitioned into regions where the value function is the…
We obtain the posterior distribution of a random process conditioned on observing the empirical frequencies of a finite sample path. We find under a rather broad assumption on the "dependence structure" of the process, {\em c.f.}…
A discrete time stochastic model for a multiagent system given in terms of a large collection of interacting Markov chains is studied. The evolution of the interacting particles is described through a time inhomogeneous transition…
We consider a type of Markov property for set-indexed processes which is satisfied by all processes with independent increments and which allows us to introduce a transition system theory leading to the construction of the process. A…
The classical Donsker weak invariance principle is extended to a Besov spaces framework. Polygonal line processes build from partial sums of stationary martingale differences as well independent and identically distributed random variables…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
Motivated by applications in systems biology, we seek a probabilistic framework based on Markov processes to represent intracellular processes. We review the formal relationships between different stochastic models referred to in the…
Markov Decision Processes (MDPs) are stochastic optimization problems that model situations where a decision maker controls a system based on its state. Partially observed Markov decision processes (POMDPs) are generalizations of MDPs where…
Stochastic transitions between discrete microscopic states play an important role in many physical and biological systems. Often, these transitions lead to fluctuations on a macroscopic scale. A classic example from neuroscience is the…
The paper deals with a certain class of random evolutions. We develop a construction that yields an invariant measure for a continuous-time Markov process with random transitions. The approach is based on a particular way of constructing…
We consider a finite-state, continuous-time Markov process, represented in the "linear framework" by a directed graph with labelled edges which specifies the infinitesimal generator of the process. If the graph is strongly connected, the…
We present limit theorems for a sequence of Piecewise Deterministic Markov Processes (PDMPs) taking values in a separable Hilbert space. This class of processes provides a rigorous framework for stochastic spatial models in which discrete…
The reduction of a continuous Markov process with multiple metastable states to a discrete rate process is investigated in the presence of slow time dependent parameters such as periodic external forces or slowly fluctuating barrier…
In the continuity of a recent paper ([6]), dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact…
This PhD thesis deals with the Markov picture of developed turbulence from the theoretical point of view. The thesis consists of two parts. The first part introduces stochastic thermodynamics, the second part aims at transferring the…
Because failures in distribution systems caused by extreme weather events directly result in consumers' outages, this paper proposes a state-based decision-making model with the objective of mitigating loss of load to improve the…
When two Markov operators commute, it suggests that we can couple two copies of one of the corresponding processes. We explicitly construct a number of couplings of this type for a commuting family of Markov processes on the set of…