Related papers: Piecewise deterministic Markov processes driven by…
We introduce and test an algorithm that adaptively estimates large deviation functions characterizing the fluctuations of additive functionals of Markov processes in the long-time limit. These functions play an important role for predicting…
For a class of piecewise deterministic Markov processes, the supports of the invariant measures are characterized. This is based on the analysis of controllability properties of an associated deterministic control system. Its invariant…
The time it takes the fastest searcher out of $N\gg1$ searchers to find a target determines the timescale of many physical, chemical, and biological processes. This time is called an extreme first passage time (FPT) and is typically much…
This paper deals with the long run average continuous control problem of piecewise deterministic Markov processes (PDMP's) taking values in a general Borel space and with compact action space depending on the state variable. The control…
The use of stochastic models, in effect piecewise deterministic Markov processes (PDMP), has become increasingly popular especially for the modeling of chemical reactions and cell biophysics. Yet, exact simulation methods, for the…
We revisit closed-loop performance guarantees for Model Predictive Control in the deterministic and stochastic cases, which extend to novel performance results applicable to receding horizon control of Partially Observable Markov Decision…
This paper presents with justifications a technique that is useful for the study of piecewise deterministic Markov decision processes (PDMDPs) with general policies and unbounded transition intensities. This technique produces an auxiliary…
We study the fluctuations of systems modeled by Markov jump processes with periodic generators. We focus on observables defined through time-periodic functions of the system's states or transitions. Using large deviation theory, canonical…
We introduce a novel type of random perturbation for the classical Lorenz flow in order to better model phenomena slowly varying in time such as anthropogenic forcing in climatology and prove stochastic stability for the unperturbed flow.…
The large deviation principle on phase space is proved for a class of Markov processes known as random population dynamics with catastrophes. In the paper we study the process which corresponds to the random population dynamics with linear…
In the real world, planning is often challenged by distribution shifts. As such, a model of the environment obtained under one set of conditions may no longer remain valid as the distribution of states or the environment dynamics change,…
This paper investigates the random horizon optimal stopping problem for measure-valued piecewise deterministic Markov processes (PDMPs). This is motivated by population dynamics applications, when one wants to monitor some characteristics…
Markovian diffusion processes yield a system of conservation laws which couple various conditional expectation values (local moments). Solutions of that closed system of deterministic partial differential equations stand for a regular…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
An extension of non-deterministic processes driven by the random telegraph signal is introduced in the framework of "piecewise deterministic Markov processes" [Davis], including a broader category of random systems. The corresponding…
We are concerned with the absolute continuity of stationary distributions corresponding to some piecewise deterministic Markov process, being typically encountered in biological models. The process under investigation involves a…
Switching dynamical systems provide a powerful, interpretable modeling framework for inference in time-series data in, e.g., the natural sciences or engineering applications. Since many areas, such as biology or discrete-event systems, are…
We develop a general theory dealing with stochastic models for dynamical systems that are governed by various nonlinear, ordinary or partial differential, equations. In particular, we address the problem how flows in the random medium…
Markov Decision Processes (Mdps) form a versatile framework used to model a wide range of optimization problems. The Mdp model consists of sets of states, actions, time steps, rewards, and probability transitions. When in a given state and…
Piecewise-deterministic Markov process (PDMP) samplers constitute a state-of-the-art Markov chain Monte Carlo paradigm in Bayesian computation, with examples including the zig-zag and bouncy particle sampler (bps). Recent work on the…