Related papers: Piecewise Deterministic Markov Processes and their…
Partially observable Markov decision processes (POMDPs) are standard models for dynamic systems with probabilistic and nondeterministic behaviour in uncertain environments. We prove that in POMDPs with long-run average objective, the…
We give a new, two-step approach to prove existence of finite invariant measures for a given Markovian semigroup. First, we identify a convenient auxiliary measure and then we prove conditions equivalent to the existence of an invariant…
We study properties of a piecewise deterministic Markov process modeling the changes in concentration of specific antibodies. The evolution of densities of the process is described by a stochastic semigroup. The long-time behaviour of this…
This paper introduces a novel stochastic framework for modelling tax evasion dynamics by extending the deterministic model of Bertotti and Modanese (2018) through the use of Piecewise Deterministic Markov Processes (PDMPs). A key limitation…
Determinantal point processes (DPPs) are probabilistic models for repulsion. When used to represent the occurrence of random subsets of a finite base set, DPPs allow to model global negative associations in a mathematically elegant and…
The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…
This article provides an introductory tutorial on structural results in partially observed Markov decision processes (POMDPs). Typically, computing the optimal policy of a POMDP is computationally intractable. We use lattice program- ming…
We study long run average behavior of generalized semi-Markov processes with both fixed-delay events as well as variable-delay events. We show that allowing two fixed-delay events and one variable-delay event may cause an unstable behavior…
In this short note we consider semi-Markov processes satisfying the condition of direction-time independence (Markov renewal processes). We derive large deviation principles and fluctuation theorems for the empirical current and the…
Noisy sensing, imperfect control, and environment changes are defining characteristics of many real-world robot tasks. The partially observable Markov decision process (POMDP) provides a principled mathematical framework for modeling and…
We provide sufficient conditions for the existence of invariant probability measures for generic stochastic differential equations with finite time delay. This is achieved by means of the Krylov-Bogoliubov method. Furthermore, we focus on…
It is well known that the space of invariant probability measures for transitive sub-shifts of finite type is a Poulsen simplex. In this article we prove that in the non-compact setting, for a large family of transitive countable Markov…
In this work, we are concerned with existence and uniqueness of invariant measures for path-dependent random diffusions and their time discretizations. The random diffusion here means a diffusion process living in a random environment…
We consider Markov decision processes (MDP) as generators of sequences of probability distributions over states. A probability distribution is p-synchronizing if the probability mass is at least p in a single state, or in a given set of…
A piecewise-deterministic Markov process is a stochastic process whose behavior is governed by an ordinary differential equation punctuated by random jumps occurring at random times. We focus on the nonparametric estimation problem of the…
Uncertain partially observable Markov decision processes (uPOMDPs) allow the probabilistic transition and observation functions of standard POMDPs to belong to a so-called uncertainty set. Such uncertainty, referred to as epistemic…
In this paper, we are interested in the exact simulation of a class of Piecewise Deterministic Markov Processes (PDMP). We show how to perform efficient thinning algorithms depending on the jump rate bound. For different types of jump rate…
Multivariate data sources with components of different information value seem to appear frequently in practice. Models in which the components change their homogeneity at different times are of significant importance. The fact whether any…
This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of…
Markov decision processes (MDPs) are a popular model for decision-making in the presence of uncertainty. The conventional view of MDPs in verification treats them as state transformers with probabilities defined over sequences of states and…