Related papers: Mixing and observation for Markov operator cocycle…
This paper compiles several aspects of the dynamics of stochastic approximation algorithms with Markov iterate-dependent noise when the iterates are not known to be stable beforehand. We achieve the same by extending the lock-in probability…
Exact inference for hidden Markov models requires the evaluation of all distributions of interest - filtering, prediction, smoothing and likelihood - with a finite computational effort. This article provides sufficient conditions for exact…
We study the so-called two-time-scale stochastic approximation, a simulation-based approach for finding the roots of two coupled nonlinear operators. Our focus is to characterize its finite-time performance in a Markov setting, which often…
We define a class of Markovian parallel dynamics for spin systems on arbitrary graphs with nearest neighbor interaction described by a Hamiltonian function $H(\sigma)$. These dynamics turn out to be reversible and their stationary measure…
Using straightforward linear algebra we derive response operators describing the impact of small perturbations to finite state Markov processes. The results can be used for studying empirically constructed - e.g. from observations or…
The stochastic generators of Markov-regular operator cocycles on symmetric Fock space are studied in a variety of cases: positive cocycles, projection cocycles, and partially isometric cocycles. Moreover a class of transformations of…
In this work, we study dynamic programming (DP) algorithms for partially observable Markov decision processes with jointly continuous and discrete state-spaces. We consider a class of stochastic systems which have coupled discrete and…
When describing the effective dynamics of an observable in a many-body system, the repeated randomness assumption, which states that the system returns in a short time to a maximum entropy state, is a crucial hypothesis to guarantee that…
We give a fairly complete characterization of the exact components of a large class of uniformly expanding Markov maps of $\mathbb{R}$. Using this result, for a class of $\mathbb{Z}$-invariant maps and finite modifications thereof, we prove…
In this article, we propose a Lyapunov stability approach to analyze the convergence of the density operator of a quantum system. In analog to the classical probability measure for Markovian processes, we show that the set of invariant…
This paper considers maximum likelihood (ML) estimation in a large class of models with hidden Markov regimes. We investigate consistency of the ML estimator and local asymptotic normality for the models under general conditions which allow…
In this paper we develop the theory of {\it polymorphisms} of measure spaces, which is a generalization of the theory of measure-preserving transformations; we describe the main notions and discuss relations to the theory of Markov…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…
The paper presents a method by which the mean field dynamics of a population of dynamical systems with parameter diversity and global coupling can be described in terms of a few macroscopic degrees of freedom. The method applies to…
Self-similarity of systems is very popular and intensively developing field during last decades. To this field belong so-called stable distributions and their generalization. In Klebanov and Sl\'amov\'a (2014) there was given an approach to…
We define the concept of an `open' Markov process, a continuous-time Markov chain equipped with specified boundary states through which probability can flow in and out of the system. External couplings which fix the probabilities of…
We give a succinct and self-contained description of the synchronized motion on networks of mutually coupled oscillators. Usually, the stability criterion for the stability of synchronized motion is obtained in terms of Lyapunov exponents.…
We consider additive functionals of Markov processes in continuous time with general (metric) state spaces. We derive concentration bounds for their exponential moments and moments of finite order. Applications include diffusions,…
We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes…
The hitting and mixing times are two fundamental quantities associated with Markov chains. In Peres and Sousi[PS2015] and Oliveira[Oli2012], the authors show that the mixing times and "worst-case" hitting times of reversible Markov chains…