Related papers: Markov bases: a 25 year update
We study weak convergence of empirical processes of dependent data $(X_i)_{i\geq0}$, indexed by classes of functions. Our results are especially suitable for data arising from dynamical systems and Markov chains, where the central limit…
We consider Markov decision processes (MDPs) in which the transition probabilities and rewards belong to an uncertainty set parametrized by a collection of random variables. The probability distributions for these random parameters are…
Sequential scaling is a prominent inference-time scaling paradigm, yet its performance improvements are typically modest and not well understood, largely due to the prevalence of heuristic, non-principled approaches that obscure clear…
Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…
The large deviations at Level 2.5 are applied to Markov processes with absorbing states in order to obtain the explicit extinction rate of metastable quasi-stationary states in terms of their empirical time-averaged density and of their…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…
We consider Markov chains that obey the following general non-linear state space model: $\Phi_{k+1} = F(\Phi_k, \alpha(\Phi_k, U_{k+1}))$ where the function $F$ is $C^1$ while $\alpha$ is typically discontinuous and $\{U_k: k \in…
Markov chain Monte Carlo (MCMC) algorithms provide a very general recipe for estimating properties of complicated distributions. While their use has become commonplace and there is a large literature on MCMC theory and practice, MCMC users…
We consider the filtering of continuous-time finite-state hidden Markov models, where the rate and observation matrices depend on unknown time-dependent parameters, for which no prior or stochastic model is available. We quantify and…
In this paper we establish spatial central limit theorems for a large class of supercritical branching Markov processes with general spatial-dependent branching mechanisms. These are generalizations of the spatial central limit theorems…
The main goal of this text is comprehensive study of time homogeneous Markov chains on the real line whose drift tends to zero at infinity, we call such processes Markov chains with asymptotically zero drift. Traditionally this topic is…
The methodology based on the random walk processes is adapted and applied to a comprehensive analysis of the statistical properties of the probability fluxes. To this aim we define a simple model of the Markovian stochastic dynamics on a…
We derive a Markov basis consisting of moves of degree at most three for two-state toric homogeneous Markov chain model of arbitrary length without parameters for initial states. Our basis consists of moves of degree three and degree one,…
Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain…
Computing optimal conditional reachability probabilities in Markov decision processes (MDPs) is tractable by a reduction to reachability probabilities. Yet, this reduction yields cyclic, challenging MDPs that are often notoriously hard to…
Positive dependencies have been compared in the literature under rather strong assumptions such as equality of conditional distributions, exchangeability, or stationarity. We establish supermodular ordering results for distributions that…
Diaconis and Sturmfels introduced an influential method to construct Markov chains using commutative algebra. One major point of their method is that infinite families of graphs are simultaneously proved to be connected by a single…
The present paper focuses on the problem of sampling from a given target distribution $\pi$ defined on some general state space. To this end, we introduce a novel class of non-reversible Markov chains, each chain being defined on an…
Conditional diffusion models serve as the foundation of modern image synthesis and find extensive application in fields like computational biology and reinforcement learning. In these applications, conditional diffusion models incorporate…
Recent works in Learning-Based Model Predictive Control of dynamical systems show impressive sample complexity performances using criteria from Information Theory to accelerate the learning procedure. However, the sequential exploration…