Related papers: Skolem and Positivity Completeness of Ergodic Mark…

A stable-like Markov chain is a time-homogeneous Markov chain on the real line with the transition kernel $p(x,dy)=f_x(y-x)dy$, where the density functions $f_x(y)$, for large $|y|$, have a power-law decay with exponent $\alpha(x)+1$, where…

The fundamental model-checking problem, given as input a model and a specification, asks for the algorithmic verification of whether the model satisfies the specification. Two classical models for reactive systems are graphs and Markov…

Risk-sensitive control balances performance with resilience to unlikely events in uncertain systems. This paper introduces ergodic-risk criteria, which capture long-term cumulative risks through probabilistic limit theorems. By ensuring the…

Upon the introduction of the Metropolis algorithm, the question of how many steps in the Markov chain were needed to achieve convergence to stationarity became apparent. The convergence was rather slow, i.e. for a process on $n$ states the…

Markov chain analysis is a key technique in formal verification. A practical obstacle is that all probabilities in Markov models need to be known. However, system quantities such as failure rates or packet loss ratios, etc. are often not --…

We show that Neural ODEs, an emerging class of time-continuous neural networks, can be verified by solving a set of global-optimization problems. For this purpose, we introduce Stochastic Lagrangian Reachability (SLR), an abstraction-based…

We derive explicit upper bounds for the $\bar{d}$-distance between a chain of infinite order and its canonical $k$-steps Markov approximation. Our proof is entirely constructive and involves a "coupling from the past" argument. The new…

Parametric Markov chains (pMC) are used to model probabilistic systems with unknown or partially known probabilities. Although (universal) pMC verification for reachability properties is known to be coETR-complete, there have been efforts…

Motivated by a derandomization of Markov chain Monte Carlo (MCMC), this paper investigates deterministic random walks, which is a deterministic process analogous to a random walk. While there are several progresses on the analysis of the…

Synchronous dynamic systems are well-established models that have been used to capture a range of phenomena in networks, including opinion diffusion, spread of disease and product adoption. We study the three most notable problems in…

In this paper, we address the problem of finding the best ergodic or Birkhoff averages in the mean ergodic theorem to ensure rapid convergence to a desired value, using graph filters. Our approach begins by representing a function on the…

We extend elliptical slice sampling, a Markov chain transition kernel suggested in Murray, Adams and MacKay 2010, to infinite-dimensional separable Hilbert spaces and discuss its well-definedness. We point to a regularity requirement,…

We prove an ergodic theorem for Markov chains indexed by the Ulam-Harris-Neveu tree over large subsets with arbitrary shape under two assumptions: with high probability, two vertices in the large subset are far from each other and have…

We study the limiting behavior of a random dynamic system driven by a stochastic chain. Our main interest is in the chains that are not necessarily ergodic but rather decomposable into ergodic classes. To investigate the conditions under…

Recursive stochastic algorithms have gained significant attention in the recent past due to data driven applications. Examples include stochastic gradient descent for solving large-scale optimization problems and empirical dynamic…

This work discusses the reachability analysis (RA) of Max-Plus Linear (MPL) systems, a class of continuous-space, discrete-event models defined over the max-plus algebra. Given the initial and target sets, we develop algorithms to verify…

A nonlinear Markov chain is a discrete time stochastic process whose transitions depend on both the current state and the current distribution of the process. The nonlinear Markov chain over a infinite state space can be identified by a…

This article presents several results establishing connections be- tween Markov chains and dynamical systems, from the point of view of open systems in physics. We show how all Markov chains can be understood as the information on one…

A classic reachability problem for safety of dynamic systems is to compute the set of initial states from which the state trajectory is guaranteed to stay inside a given constraint set over a given time horizon. In this paper, we leverage…

Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…