Related papers: Conditioning continuous-time Markov processes by g…
Consider a system $X = ((x_\xi(t)), \xi \in \Omega_N)_{t \geq 0}$ of interacting Fleming-Viot diffusions with mutation and selection which is a strong Markov process with continuous paths and state space $(\CP(\I))^{\Omega_N}$, where $\I$…
In this paper, we consider semi-Markov processes whose transition times and transition probabilities depend on a small parameter $\varepsilon$. Understanding the asymptotic behavior of such processes is needed in order to study the…
We study the asymptotic hitting time $\tau^{(n)}$ of a family of Markov processes $X^{(n)}$ to a target set $G^{(n)}$ when the process starts from a trap defined by very general properties. We give an explicit description of the law of…
We propose a method for approximating solutions to optimization problems involving the global stability properties of parameter-dependent continuous-time autonomous dynamical systems. The method relies on an approximation of the…
This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…
The evolution of a continuous time Markov process with a finite number of states is usually calculated by the Master equation - a linear differential equations with a singular generator matrix. We derive a general method for reducing the…
A comparison theorem for state-dependent regime-switching diffusion processes is established, which enables us to control pathwisely the evolution of the state-dependent switching component simply by Markov chains. Moreover, a sharp…
Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition…
We study a class of Piecewise Deterministic Markov Processes with state space Rd x E where E is a finite set. The continuous component evolves according to a smooth vector field that is switched at the jump times of the discrete coordinate.…
In this work, we generalize the concept of bisimulation metric in order to metrize the behaviour of continuous-time processes. Similarly to what is done for discrete-time systems, we follow two approaches and show that they coincide: as a…
Probabilistic generative models based on measure transport, such as diffusion and flow-based models, are often formulated in the language of Markovian stochastic dynamics, where the choice of the underlying process impacts both algorithmic…
In this paper we consider a reduced-form intensity-based credit risk model with a hidden Markov state process. A filtering method is proposed for extracting the underlying state given the observation processes. The method may be applied to…
In the field of large deviations for stochastic dynamics, the canonical conditioning of a given Markov process with respect to a given time-local trajectory observable over a large time-window has attracted a lot of interest recently. In…
Consider a sequence of Markov processes $X^1, X^2,...$ with state space $E$, where $X^N$ has a strong drift to $D \subseteq E$, such that $\Phi(X^N)$ is slow for some appropriate $\Phi: E\to D$. Using the method of martingale problems, we…
Complex physical dynamics can often be modeled as a Markov jump process between mesoscopic configurations. When jumps between mesoscopic states are mediated by thermodynamic reservoirs, the time-irreversibility of the jump process is a…
The expansion of a stochastic Liouville equation for the coupled evolution of a quantum system and an Ornstein-Uhlenbeck process into a hierarchy of coupled differential equations is a useful technique that simplifies the simulation of…
Examples of stochastic processes whose state space representations involve functions of an integral type structure $$I_{t}^{(a,b)}:=\int_{0}^{t}b(Y_{s})e^{-\int_{s}^{t}a(Y_{r})dr}ds, \quad t\ge 0$$ are studied under an ergodic…
In order to study the stochastic Markov processes conditioned on a specific value of a time-integrated observable, the concept of ensembles of trajectories has been recently used extensively. In this paper, we consider a generic…
We investigate piecewise deterministic Markov processes (PDMP), where the deterministic dynamics follows a scalar conservation law and random jumps in the system are characterized by changes in the flux function. We show under which…
It is known that state-dependent, multi-step Lyapunov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the "fluid model" approach to stability of…