Related papers: Contractive Markov systems II
We consider an absorbing Markov decision process with Borel state and action spaces. We study conditions under which the MDP is uniformly absorbing and the set of occupation measures of the MDP is compact in the usual weak topology. These…
We consider the invariant measure of a homogeneous continuous- time Markov process in the quarter-plane. The basic solutions of the global balance equation are the geometric distributions. We first show that the invariant measure can not be…
Ergodic properties of rational maps are studied, generalising the work of F.\ Ledrappier. A new construction allows for simpler proofs of stronger results. Very general conformal measures are considered. Equivalent conditions are given for…
Understanding and predicting how complex systems respond to external perturbations is a central challenge in nonequilibrium statistical physics. Here we consider continuous-time Markov networks, which we subject to perturbations along a…
The hypercontractivity is proved for the Markov semigroup associated to a class of finite/infinite dimensional stochastic Hamiltonian systems. Consequently, the Markov semigroup is exponentially convergent to the invariant probability…
We study ergodic properties of some Markov chains models in random environments when the random Markov kernels that define the dynamic satisfy some usual drift and small set conditions but with random coefficients. In particular, we adapt a…
A general setting for nested subdivisions of a bounded real set into intervals defining the digits $X_1,X_2,...$ of a random variable $X$ with a probability density function $f$ is considered. Under the weak condition that $f$ is almost…
For time-inconsistent stochastic controls in discrete time and finite horizon, an open problem in Bj\"ork and Murgoci (Finance Stoch, 2014) is the existence of an equilibrium control. A nonrandomized Borel measurable Markov equilibrium…
This paper, based on the compactness-continuity and finite value conditions, establishes the sufficiency of the class of stationary policies out of the general class of history-dependent ones for a constrained continuous-time Markov…
Let $\{X_n\}$ be a stationary and ergodic time series taking values from a finite or countably infinite set ${\cal X}$. Assume that the distribution of the process is otherwise unknown. We propose a sequence of stopping times $\lambda_n$…
Dynamical systems that are contracting on a subspace are said to be semicontracting. Semicontraction theory is a useful tool in the study of consensus algorithms and dynamical flow systems such as Markov chains. To develop a comprehensive…
In this paper, we develop the theory of contractive Markov systems initiated in \cite{Wer1}. We construct a coding map for such systems and investigate some of its properties.
We show that nonequilibrium dynamics can play a constructive role in unsupervised machine learning by inducing the spontaneous emergence of latent-state cycles. We introduce a model in which visible and hidden variables interact through two…
This paper contains two parts. In the first part, we study the ergodicity of periodic measures of random dynamical systems on a separable Banach space. We obtain that the periodic measure of the continuous time skew-product dynamical system…
We analyse the structure of imprecise Markov chains and study their convergence by means of accessibility relations. We first identify the sets of states, so-called minimal permanent classes, that are the minimal sets capable of containing…
In this work, we present a general method to establish properties of multi-dimensional continuous-time Markov chains representing stochastic reaction networks. This method consists of grouping states together (via a partition of the state…
Consider a topological dynamical system where the group is abelian and the topologies are locally compact and second-countable. Given an invariant measure for this system, we show that if its dynamical spectrum is contained in some Borel…
We analyze dynamical systems subjected to an additive noise and their deterministic limit. In this work, we will introduce a notion by which a stochastic system has something like a Markov partition for deterministic systems. For a chosen…
Via operator theoretic methods, we formalize the concentration phenomenon for a given observable `$r$' of a discrete time Markov chain with `$\mu_{\pi}$' as invariant ergodic measure, possibly having support on an unbounded state space. The…
We consider additive functionals of Markov processes in continuous time with general (metric) state spaces. We derive concentration bounds for their exponential moments and moments of finite order. Applications include diffusions,…