Related papers: Probabilistic and Piecewise Deterministic models i…
Markov decision processes (MDPs) in queues and networks have been an interesting topic in many practical areas since the 1960s. This paper provides a detailed overview on this topic and tracks the evolution of many basic results. Also, this…
In the development of stochastic integration and the theory of semimartingales, Markov processes have been a constant source of inspiration. Despite this historical interweaving, it turned out that semimartingales should be considered the…
An extension of non-deterministic processes driven by the random telegraph signal is introduced in the framework of "piecewise deterministic Markov processes" [Davis], including a broader category of random systems. The corresponding…
Dynamic Movement Primitives have successfully been used to realize imitation learning, trial-and-error learning, reinforce- ment learning, movement recognition and segmentation and control. Because of this they have become a popular…
We study a subclass of POMDPs, called Deterministic POMDPs, that is characterized by deterministic actions and observations. These models do not provide the same generality of POMDPs yet they capture a number of interesting and challenging…
Stochastic dynamical systems arise naturally across nearly all areas of science and engineering. Typically, a dynamical system model is based on some prior knowledge about the underlying dynamics of interest in which probabilistic features…
The synthesis problem for partially observable Markov decision processes (POMDPs) is to compute a policy that satisfies a given specification. Such policies have to take the full execution history of a POMDP into account, rendering the…
We aim at characterizing the asymptotic behavior of value functions in the control of piece-wise deterministic Markov processes (PDMP) of switch type under nonexpansive assumptions. For a particular class of processes inspired by temperate…
This note provides several recent progresses in the study of long time behavior of Markov processes. The examples presented below are related to other scientific fields as PDE's, physics or biology. The involved mathematical tools as…
We are interested in modelling Darwinian evolution, resulting from the interplay of phenotypic variation and natural selection through ecological interactions. Our models are rooted in the microscopic, stochastic description of a population…
The paper discusses a family of Markov processes that represent many particle systems, and their limiting behaviour when the number of particles go to infinity. The first part concerns model of biological systems: a model for sympatric…
We study algorithms to analyze a particular class of Markov population processes that is often used in epidemiology. More specifically, Markov binomial chains are the model that arises from stochastic time-discretizations of classical…
In this paper, a condition-based imperfect maintenance model based on piecewise deterministic Markov process (PDMP) is constructed. The degradation of the system includes two types: natural degradation and random shocks. The natural…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
We cast episodic Markov decision process (MDP) planning as Bayesian inference over policies. A policy is treated as the latent variable and is assigned an unnormalized probability of optimality that is monotone in its expected return,…
This paper studies parametric Markov decision processes (pMDPs), an extension to Markov decision processes (MDPs) where transitions probabilities are described by polynomials over a finite set of parameters. Fixing values for all parameters…
In this paper, a class of piecewise deterministic Markov processes with underlying fast dynamic is studied. Using a "penalty method" , an averaging result is obtained when the underlying dynamic is infinitely accelerated. The features of…
The large deviation principle on phase space is proved for a class of Markov processes known as random population dynamics with catastrophes. In the paper we study the process which corresponds to the random population dynamics with linear…
We prove a Large Deviation Principle for Piecewise Deterministic Markov Processes (PDMPs). This is an asymptotic estimate for the probability of a trajectory in the large size limit. Explicit Euler-Lagrange equations are determined for…
General purpose intelligent learning agents cycle through (complex,non-MDP) sequences of observations, actions, and rewards. On the other hand, reinforcement learning is well-developed for small finite state Markov Decision Processes…