Related papers: On characterisation of Markov processes via martin…
This paper focuses on optimizing probabilities of events of interest defined over general controlled discrete-time Markov processes. It is shown that the optimization over a wide class of $\omega$-regular properties can be reduced to the…
We consider a class of stochastic control problems where the state process is a probability measure-valued process satisfying an additional martingale condition on its dynamics, called measure-valued martingales (MVMs). We establish the…
We construct a class of discontinuous superprocesses with dependent spatial motion and general branching mechanism. The process arises as the weak limit of critical interacting-branching particle systems where the spatial motions of the…
Adaptive Markov chain Monte Carlo (MCMC) algorithms, which automatically tune their parameters based on past samples, have proved extremely useful in practice. The self-tuning mechanism makes them `non-Markovian', which means that their…
Most of Markov Chain Monte Carlo (MCMC) and sequential Monte Carlo (SMC) algorithms in existing probabilistic programming systems suboptimally use only model priors as proposal distributions. In this work, we describe an approach for…
Euclidean Markov decision processes are a powerful tool for modeling control problems under uncertainty over continuous domains. Finite state imprecise, Markov decision processes can be used to approximate the behavior of these infinite…
Termination is one of the basic liveness properties, and we study the termination problem for probabilistic programs with real-valued variables. Previous works focused on the qualitative problem that asks whether an input program terminates…
We provide a systematic, thorough treatment of the foundations of probability theory and stochastic processes along the lines of E. Bishop's constructive analysis. Every existence result presented shall be a construction; and the input…
We construct a family of self-similar Markov martingales with given marginal distributions. This construction uses the self-similarity and Markov property of a reference process to produce a family of Markov processes that possess the same…
We establish general theorems quantifying the notion of recurrence --- through an estimation of the moments of passage times --- for irreducible continuous-time Markov chains on countably infinite state spaces. Sharp conditions of…
The paper introduces and investigates the natural extension to the path-dependent setup of the usual concept of canonical Markov class introduced by Dynkin and which is at the basis of the theory of Markov processes. That extension, indexed…
The deterministic analog of the Markov property of a time-homogeneous Markov process is the semigroup property of solutions of an autonomous differential equation. The semigroup property arises naturally when the solutions of a differential…
We propose an alternative approach for solving a number of well-studied optimal stopping problems for L\'evy processes. Instead of the usual method of guess-and-verify based on martingale properties of the value function, we suggest a more…
We offer a new proof of the classical law of large numbers for a general class of branching Markov processes based on the asymptotic behaviour of the moments developed in \cite{bmoments, gonzalez2022erratum}. Moreover, we show that the law…
We give a bare-hands approach to the martingale representation theorem for integer valued random measures, which allows for a wide class of infinite activity jump processes, as well as all processes with well-ordered jumps.
We introduce a class of so called Markovian marginals, which gives a natural framework for constructing solutions to the quantum marginal problem. We consider a set of marginals that possess a certain internal quantum Markov chain…
Markov chain Monte Carlo (MCMC) algorithms provide a very general recipe for estimating properties of complicated distributions. While their use has become commonplace and there is a large literature on MCMC theory and practice, MCMC users…
We explore two notions of stationary processes. The first is called a random-step Markov process in which the stationary process of states, $(X_i)_{i \in \mathbb{Z}}$ has a stationary coupling with an independent process on the positive…
We consider the Markov chain approximations for singular stable-like processes. First we obtain properties of some Markov chains. Then we construct the approximating Markov chains and give a necessary condition for weak convergence of these…
Markov chain Monte Carlo (MCMC) algorithms are based on the construction of a Markov chain with transition probabilities leaving invariant a probability distribution of interest. In this work, we look at these transition probabilities as…