Related papers: A classification of bisimilarities for general Mar…
We present an investigation of stochastic evolution in which a family of evolution equations in $L^1$ are driven by continuous-time Markov processes. These are examples of so-called piecewise deterministic Markov processes (PDMP's) on the…
A Markov decision process (MDP) is a state-based dynamical system capable of describing probabilistic behaviour with rewards. In this paper, we view MDPs as coalgebras living in the category of analytic spaces, a very general class of…
We prove that the relation of bisimilarity between countable labelled transition systems is $\Sigma_1^1$-complete (hence not Borel), by reducing the set of non-wellorders over the natural numbers continuously to it. This has an impact on…
This paper studies a new and more general axiomatization than one presented previously for preference on likelihood gambles. Likelihood gambles describe actions in a situation where a decision maker knows multiple probabilistic models and a…
It is common, when dealing with quantum processes involving a subsystem of a much larger composite closed system, to treat them as effectively memory-less (Markovian). While open systems theory tells us that non-Markovian processes should…
Individual events at high-energy colliders like the LHC can be represented by a sequence of measurements, or 'point patterns' in an observable space. Starting from this data representation, we build a simple Bayesian probabilistic model for…
In this paper we study the equivalence of nondeterministic automata pairing the concept of a bisimulation with the recently introduced concept of a uniform relation. In this symbiosis, uniform relations serve as equivalence relations which…
We study a probabilistic variant of binary session types that relate to a class of Finite-State Markov Chains. The probability annotations in session types enable the reasoning on the probability that a session terminates successfully, for…
Empirical processes for stationary, causal sequences are considered. We establish empirical central limit theorems for classes of indicators of left half lines, absolutely continuous functions and piecewise differentiable functions. Sample…
The generalized labeled multi-Bernoulli (GLMB) is a family of tractable models that alleviates the limitations of the Poisson family in dynamic Bayesian inference of point processes. In this paper, we derive closed form expressions for the…
The Markov assumption in Markov Decision Processes (MDPs) is fundamental in reinforcement learning, influencing both theoretical research and practical applications. Existing methods that rely on the Bellman equation benefit tremendously…
Starting from the forward and backward infinitesimal generators of bilateral, time-homogeneous Markov processes, the self-adjoint Hamiltonians of the generalized Schroedinger equations are first introduced by means of suitable Doob…
We provide a rather explicit formula for the resolvent of a~concatenation of $N$ processes in terms of their exit laws and certain probability measures characterizing the way the processes are concatenated. As an application, we prove an…
Stochastic games are an important class of problems that generalize Markov decision processes to game theoretic scenarios. We consider finite state two-player zero-sum stochastic games over an infinite time horizon with discounted rewards.…
Stationary ergodic processes with finite alphabets are estimated by finite memory processes from a sample, an n-length realization of the process, where the memory depth of the estimator process is also estimated from the sample using…
We introduce and study randomized sequential importance sampling algorithms for estimating the number of perfect matchings in bipartite graphs. In analyzing their performance, we establish various non-standard central limit theorems. We…
Matching games is a novel matching model introduced by Garrido-Lucero and Laraki, in which agents' utilities are endogenously determined as the outcome of a strategic game they play simultaneously with the matching process. Matching games…
We review some recent results of quantitative long-time convergence for the law of a killed Markov process conditioned to survival toward a quasi-stationary distribution, and on the analogous question for the particle systems used in…
A powerful tool for studying long-term convergence of a Markov process to its stationary distribution is a Lyapunov function. In some sense, this is a substitute for eigenfunctions. For a stochastically ordered Markov process on the…
This note considers the notion of divergence-preserving branching bisimilarity. It briefly surveys results pertaining to the notion that have been obtained in the past one-and-a-half decade, discusses its role in the study of expressiveness…