Related papers: Evidential Decision Theory via Partial Markov Cate…
We introduce Markov substitute processes, a new model at the crossroad of statistics and formal grammars, and prove its main property : Markov substitute processes with a given support form an exponential family.
The paper provides an overview of the theory and applications of risk-sensitive Markov decision processes. The term 'risk-sensitive' refers here to the use of the Optimized Certainty Equivalent as a means to measure expectation and risk.…
Continuous-time Markov decision processes are an important class of models in a wide range of applications, ranging from cyber-physical systems to synthetic biology. A central problem is how to devise a policy to control the system in order…
This paper addresses the challenge of a particular class of noisy state observations in Markov Decision Processes (MDPs), a common issue in various real-world applications. We focus on modeling this uncertainty through a confusion matrix…
In this article we propose a qualitative (ordinal) counterpart for the Partially Observable Markov Decision Processes model (POMDP) in which the uncertainty, as well as the preferences of the agent, are modeled by possibility distributions.…
We argue that locally Cartesian closed categories form a suitable doctrine for defining dependent type theories, including non-extensional ones. Using the theory of sketches, one may define syntactic categories for type theories in a style…
We introduce quantum Markov categories as a structure that refines and extends a synthetic approach to probability theory and information theory so that it includes quantum probability and quantum information theory. In this broader…
A new sufficient condition is proved for the existence of stochastic semigroups generated by the sum of two unbounded operators. It is applied to one-dimensional piecewise deterministic Markov processes, where we also discuss the existence…
In the paper we consider some piecewise deterministic Markov process whose continuous component evolves according to semiflows, which are switched at the jump times of a Poisson process. The associated Markov chain describes the states of…
We classify localising subcategories of the stable module category of a finite group that are closed under tensor product with simple (or, equivalently all) modules. One application is a proof of the telescope conjecture in this context.…
Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically.…
We continue the investigation of the spectral theory and exponential asymptotics of Markov processes, following Kontoyiannis and Meyn (2003). We introduce a new family of nonlinear Lyapunov drift criteria, characterizing distinct subclasses…
Synthesising verifiably correct controllers for dynamical systems is crucial for safety-critical problems. To achieve this, it is important to account for uncertainty in a robust manner, while at the same time it is often of interest to…
We consider evaluating improper priors in a formal Bayes setting according to the consequences of their use. Let $\Phi$ be a class of functions on the parameter space and consider estimating elements of $\Phi$ under quadratic loss. If the…
We introduce the concept of an imprecise Markov semigroup \(\mathbf Q\). It is a tool that allows us to represent ambiguity around both the transition probabilities and the invariant measure of a continuous-time Markov process via a…
We introduce a new definition of exponential family of Markov chains, and show that many characteristic properties of the usual exponential family of probability distributions are properly extended to Markov chains. The method of…
Even simply-defined, finite-state generators produce stochastic processes that require tracking an uncountable infinity of probabilistic features for optimal prediction. For processes generated by hidden Markov chains the consequences are…
Markov decision processes (MDPs) are a popular model for decision-making in the presence of uncertainty. The conventional view of MDPs in verification treats them as state transformers with probabilities defined over sequences of states and…
We consider stochastic thermodynamics as a theory of statistical inference for experimentally observed fluctuating time-series. To that end, we introduce a general framework for quantifying the knowledge about the dynamical state of the…
We consider prediction theory for stationary stochastic processes in continuous time. We discuss prediction using the whole (infinite) past, and using only a finite section of the past. The solutions to both these classical problems have…