Related papers: Supports of Invariant Measures for Piecewise Deter…
Sequential Monte Carlo Samplers are a class of stochastic algorithms for Monte Carlo integral estimation w.r.t. probability distributions, which combine elements of Markov chain Monte Carlo methods and importance sampling/resampling…
This paper studies Markov Decision Processes under parameter uncertainty. We adapt the distributionally robust optimization framework, and assume that the uncertain parameters are random variables following an unknown distribution, and…
We give sufficient conditions on the rates of two asymmetric exclusion processes such that the existence of a blocking invariant measure for the first implies the existence of such a measure for the second. The main tool is a coupling…
We develop a convergent variational perturbation theory for conditional probability densities of Markov processes. The power of the theory is illustrated by applying it to the diffusion of a particle in an anharmonic potential.
We study the structure of invariant measures for continuous automorphisms of compact metrizable abelian groups satisfying the descending chain condition. We show that the finitely supported invariant measures are weak-* dense in the space…
In this paper we develop the theory of {\it polymorphisms} of measure spaces, which is a generalization of the theory of measure-preserving transformations; we describe the main notions and discuss relations to the theory of Markov…
This paper presents a nonparametric method for estimating the conditional density associated to the jump rate of a piecewise-deterministic Markov process. In our framework, the estimation needs only one observation of the process within a…
We consider the expressivity of Markov rewards in sequential decision making under uncertainty. We view reward functions in Markov Decision Processes (MDPs) as a means to characterize desired behaviors of agents. Assuming desired behaviors…
The purpose of this paper is to study a Markovian metapopulation model on a directed graph with edge-supported transfers and deterministic intra-nodal population dynamics. We first state tractable stability conditions for two typical…
We study the finiteness of physical measures for skew-product transformations $F$ associated with discrete-time random dynamical systems driven by ergodic Markov chains. We develop a framework, using an independent and identically…
This paper presents two stochastic model predictive control methods for linear time-invariant systems subject to unbounded additive uncertainties. The new methods are developed by formulating the chance constraints into deterministic form,…
This paper studies parametric Markov decision processes (pMDPs), an extension to Markov decision processes (MDPs) where transitions probabilities are described by polynomials over a finite set of parameters. Fixing values for all parameters…
This paper develops a variational inference framework for control of infinite dimensional stochastic systems. We employ a measure theoretic approach which relies on the generalization of Girsanov's theorem, as well as the relation between…
Euclidean Markov decision processes are a powerful tool for modeling control problems under uncertainty over continuous domains. Finite state imprecise, Markov decision processes can be used to approximate the behavior of these infinite…
For a class of piecewise deterministic Markov processes we introduce a stochastic calculus which is a certain non-Gaussian counterpart to the classical Malliavin calculus. As an application we investigate the regularity of densities of…
We show the existence of Lebesgue-equivalent conservative and ergodic $\sigma$-finite invariant measures for a wide class of one-dimensional random maps consisting of piecewise convex maps. We also estimate the size of invariant measures…
We describe all boundedly finite measures which are invariant by Cartesian powers of an infinite measure preserving version of Chacon transformation. All such ergodic measures are products of so-called diagonal measures, which are measures…
We study discretizations of polynomial processes using finite state Markov processes satisfying suitable moment matching conditions. The states of these Markov processes together with their transition probabilities can be interpreted as…
We introduce a new tool for the quantitative characterisation of the departure form Markovianity of a given dynamical process. Our tool can be applied to a generic $N$-level system and extended straightforwardly to Gaussian…
Variance reduction methods are often needed for the reliability assessment of complex industrial systems, we focus on one variance reduction method in a given context, that is the interacting particle system method (IPS) used on piecewise…