Related papers: Measure-Valued Generators of General Piecewise Det…

Even simply-defined, finite-state generators produce stochastic processes that require tracking an uncountable infinity of probabilistic features for optimal prediction. For processes generated by hidden Markov chains the consequences are…

This paper discusses the functional stability of closed-loop Markov Chains under optimal policies resulting from a discounted optimality criterion, forming Markov Decision Processes (MDPs). We investigate the stability of MDPs in the sense…

Semi-Markov processes play an important role in the effective description of partially accessible systems in stochastic thermodynamics. They occur, for instance, in coarse-graining procedures such as state lumping and when analyzing waiting…

This paper is devoted to parameter estimation for partially observed polynomial state space models. This class includes discretely observed affine or more generally polynomial Markov processes. The polynomial structure allows for the…

Partial mean with generated regressors arises in several econometric problems, such as the distribution of potential outcomes with continuous treatments and the quantile structural function in a nonseparable triangular model. This paper…

Partially observable Markov decision processes (POMDPs) are widely used in probabilistic planning problems in which an agent interacts with an environment using noisy and imprecise sensors. We study a setting in which the sensors are only…

This paper introduces a novel stochastic framework for modelling tax evasion dynamics by extending the deterministic model of Bertotti and Modanese (2018) through the use of Piecewise Deterministic Markov Processes (PDMPs). A key limitation…

We exhibit conditions under which the flow of marginal distributions of a discontinuous semimartingale $\xi$ can be matched by a Markov process, whose infinitesimal generator is expressed in terms of the local characteristics of $\xi$. Our…

In many practical settings control decisions must be made under partial/imperfect information about the evolution of a relevant state variable. Partially Observable Markov Decision Processes (POMDPs) is a relatively well-developed framework…

We consider a distributionally robust Partially Observable Markov Decision Process (DR-POMDP), where the distribution of the transition-observation probabilities is unknown at the beginning of each decision period, but their realizations…

Many real-world decision-theoretic planning problems can be naturally modeled with discrete and continuous state Markov decision processes (DC-MDPs). While previous work has addressed automated decision-theoretic planning for DCMDPs,…

In this paper, we continue development of the theory of contractive Markov systems (CMS) initiated in \cite{Wer1}. Also, this work can be seen as a small contribution to the theory of equilibrium states. We construct an energy function on…

We study finite-state controllers (FSCs) for partially observable Markov decision processes (POMDPs) that are provably correct with respect to given specifications. The key insight is that computing (randomised) FSCs on POMDPs is equivalent…

Piecewise Deterministic Markov Processes (PDMPs) provide a powerful framework for continuous-time Monte Carlo, with the Bouncy Particle Sampler (BPS) as a prominent example. Recent advances through the Metropolised PDMP framework allow…

The synthesis problem for partially observable Markov decision processes (POMDPs) is to compute a policy that satisfies a given specification. Such policies have to take the full execution history of a POMDP into account, rendering the…

The first aims of this work are to endorse the advent of finitely additive set functions as equilibrium states and the possibility to replace the metric entropy by an upper semi-continuous map associated to a general variational principle.…

Value iteration is a well-known method of solving Markov Decision Processes (MDPs) that is simple to implement and boasts strong theoretical convergence guarantees. However, the computational cost of value iteration quickly becomes…

This paper presents a numerical method to calculate the value function for a general discounted impulse control problem for piecewise deterministic Markov processes. Our approach is based on a quantization technique for the underlying…

Score-based generative models (SGMs) have recently emerged as a promising class of generative models. However, a fundamental limitation is that their sampling process is slow due to a need for many (e.g., 2000) iterations of sequential…

Markov decision processes (MDPs) with rewards are a widespread and well-studied model for systems that make both probabilistic and nondeterministic choices. A fundamental result about MDPs is that their minimal and maximal expected rewards…