Related papers: Probabilistic and Piecewise Deterministic models i…
We extend Monte Carlo samplers based on piecewise deterministic Markov processes (PDMP samplers) by formally defining different boundary conditions such as sticky floors, soft and hard walls and teleportation portals. This allows PDMP…
Partially observable Markov decision processes (POMDPs) are a fundamental model for sequential decision-making under uncertainty. However, many verification and synthesis problems for POMDPs are undecidable or intractable. Most prominently,…
This paper investigates the random horizon optimal stopping problem for measure-valued piecewise deterministic Markov processes (PDMPs). This is motivated by population dynamics applications, when one wants to monitor some characteristics…
Partially observable Markov decision processes (POMDPs) are standard models for dynamic systems with probabilistic and nondeterministic behaviour in uncertain environments. We prove that in POMDPs with long-run average objective, the…
We construct a stochastic fluid process with an underlying piecewise deterministic Markov process (PDMP) akin to the one used in the construction of the rational arrival process (RAP), which we call the RAP-modulated fluid process. As…
The Distributed Cooperative Modeling System (DCMS) solves complex decision problems involving a lot of participants with different viewpoints by network based distributed modeling and multi-template aggregation. This thesis aims at…
We consider the problem of selecting deterministic or stochastic models for a biological, ecological, or environmental dynamical process. In most cases, one prefers either deterministic or stochastic models as candidate models based on…
Biomarker measurements obtained by blood sampling are often used as a non-invasive means of monitoring tumour progression in cancer patients. Diseases evolve dynamically over time, and studying longitudinal observations of specific…
In the paper we consider some piecewise deterministic Markov process whose continuous component evolves according to semiflows, which are switched at the jump times of a Poisson process. The associated Markov chain describes the states of…
We develop a qualitative theory of Markov Decision Processes (MDPs) and Partially Observable MDPs that can be used to model sequential decision making tasks when only qualitative information is available. Our approach is based upon an…
In adaptive systems, predictors are used to anticipate changes in the systems state or behavior that may require system adaption, e.g., changing its configuration or adjusting resource allocation. Therefore, the quality of predictors is…
The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…
In many biological systems, chemical reactions or changes in a physical state are assumed to occur instantaneously. For describing the dynamics of those systems, Markov models that require exponentially distributed inter-event times have…
We consider Markov decision processes (MDPs) which are a standard model for probabilistic systems. We focus on qualitative properties for MDPs that can express that desired behaviors of the system arise almost-surely (with probability 1) or…
Piecewise Deterministic Monte Carlo algorithms enable simulation from a posterior distribution, whilst only needing to access a sub-sample of data at each iteration. We show how they can be implemented in settings where the parameters live…
The aim of this paper is to tackle part of the program set by Diekmann et al. in their seminal paper Diekmann et al. (2001). We quote "It remains to investigate whether, and in what sense, the nonlinear determin-istic model formulation is…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
Continuous-time Markov processes over finite state-spaces are widely used to model dynamical processes in many fields of natural and social science. Here, we introduce an maximum likelihood estimator for constructing such models from data…
Variance reduction methods are often needed for the reliability assessment of complex industrial systems, we focus on one variance reduction method in a given context, that is the interacting particle system method (IPS) used on piecewise…
In this paper we define an infinite-dimensional controlled piecewise deterministic Markov process (PDMP) and we study an optimal control problem with finite time horizon and unbounded cost. This process is a coupling between a continuous…