Related papers: Fundamental Markov systems
Switching dynamical systems provide a powerful, interpretable modeling framework for inference in time-series data in, e.g., the natural sciences or engineering applications. Since many areas, such as biology or discrete-event systems, are…
A general setting for nested subdivisions of a bounded real set into intervals defining the digits $X_1,X_2,...$ of a random variable $X$ with a probability density function $f$ is considered. Under the weak condition that $f$ is almost…
We consider a class of piecewise-deterministic Markov processes where the state evolves according to a linear dynamical system. This continuous time evolution is interspersed by discrete events that occur at random times and change (reset)…
Using the tools of the Markov Decision Processes, we justify the dynamic programming approach to the optimal impulse control of deterministic dynamical systems. We prove the equivalence of the integral and differential forms of the…
We extend the theory of transience to general dynamical systems with no Markov structure assumed. This is linked to the theory of phase transitions. We also provide examples of new kinds of transient behaviour.
In this paper, we introduce the concept of random time changes in dynamical systems. The sub- ordination principle may be applied to study the long time behavior of the random time systems. We show, under certain assumptions on the class of…
We prove a weak iterated invariance principle for a large class of non-uniformly expanding random dynamical systems. In addition, we give a quenched homogenization result for fast-slow systems in the case when the fast component corresponds…
In this work, we study dynamic programming (DP) algorithms for partially observable Markov decision processes with jointly continuous and discrete state-spaces. We consider a class of stochastic systems which have coupled discrete and…
For Markov processes with absorption, we provide general criteria ensuring the existence and the exponential non-uniform convergence in total variation norm to a quasi-stationary distribution. We also characterize a subset of its domain of…
An optimal control for a dynamical system optimizes a certain objective function. Here we consider the construction of an optimal control for a stochastic dynamical system with a random structure, Poisson perturbations and random jumps,…
We study Markovian and non-Markovian behaviour of stochastic processes generated by $p$-adic random dynamical systems. Given a family of $p$-adic monomial random mappings generating a random dynamical system. Under which conditions do the…
Positive dependencies have been compared in the literature under rather strong assumptions such as equality of conditional distributions, exchangeability, or stationarity. We establish supermodular ordering results for distributions that…
In many dynamical systems in nature, the law of the dynamics changes along with the temporal evolution of the system. These changes are often associated with the occurrence of certain events. The timing of occurrence of these events…
Stochastic approximation is a framework unifying many random iterative algorithms occurring in a diverse range of applications. The stability of the process is often difficult to verify in practical applications and the process may even be…
We discuss general concept of Markov statistical dynamics in the continuum. For a class of spatial birth-and-death models, we develop a perturbative technique for the construction of statistical dynamics. Particular examples of such systems…
We study the stochastic dynamics of a system of interacting species in a stochastic environment by means of a continuous-time Markov chain with transition rates depending on the state of the environment. Models of gene regulation in systems…
We study discrete time Markov processes with periodic or open boundary conditions and with inhomogeneous rates in the bulk. The Markov matrices are given by the inhomogeneous transfer matrices introduced previously to prove the…
We study a class of Markovian systems of $N$ elements taking values in $[0,1]$ that evolve in discrete time $t$ via randomized replacement rules based on the ranks of the elements. These rank-driven processes are inspired by variants of the…
The stability of iterations of affine linear maps $\Psi_{n}(x)=A_{n}x+B_{n}$, $n=1,2,\ldots$, is studied in the presence of a Markovian environment, more precisely, for the situation when $(A_{n},B_{n})_{n\ge 1}$ is modulated by an ergodic…
We consider continuous-space, discrete-time Markov chains on $\mathbb{R}^d$, that admit a finite number $N$ of metastable states. Our main motivation for investigating these processes is to analyse random Poincar\'e maps, which describe…