Related papers: Nonexpansive Markov Operators and Random Function …
We consider positive, integral-preserving linear operators acting on $L^1$ space, known as stochastic operators or Markov operators. We show that, on finite-dimensional spaces, any stochastic operator can be approximated by a sequence of…
Motivated by applications to mathematical biology, we study the averaging problem for slow-fast systems, {\em in the case in which the fast dynamics is a stochastic process with multiple invariant measures}. We consider both the case in…
We consider the problem of average consensus in a distributed system comprising a set of nodes that can exchange information among themselves. We focus on a class of algorithms for solving such a problem whereby each node maintains a state…
In this brief note, we investigate some constructions of Lyapunov functions for stochastic discrete-time stabilizable dynamical systems, in other words, controlled Markov chains. The main question here is whether a Lyapunov function in some…
Flexible sparsity regularization means stably approximating sparse solutions of operator equations by using coefficient-dependent penalizations. We propose and analyse a general nonconvex approach in this respect, from both theoretical and…
A nonlinear Markov chain is a discrete time stochastic process whose transitions depend on both the current state and the current distribution of the process. The nonlinear Markov chain over a infinite state space can be identified by a…
This paper focuses on the invariance control problem for discrete-time switched nonlinear systems. The proposed approach computes controlled invariant sets in a finite number of iterations and directly yields a partition-based invariance…
We study invariant measures for random countable (finite or infinite) conformal iterated function systems (IFS) with arbitrary overlaps. We do not assume any type of separation condition. We prove, under a mild assumption of finite entropy,…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
The convex feasibility problem (CFP) is to find a feasible point in the intersection of finitely many convex and closed sets. If the intersection is empty then the CFP is inconsistent and a feasible point does not exist. However,…
We study Markov chains generated by iterated Lipschitz functions systems with possibly place dependent probabilities. Under general conditions, we prove uniqueness of the invariant probability measure for the associated Markov chain, by…
We give a systematic account of iterated function systems (IFS) of weak contractions of different types (Browder, Rakotch, topological). We show that the existence of attractors and asymptotically stable invariant measures, and the validity…
We study computing geometric problems on uncertain points. An uncertain point is a point that does not have a fixed location, but rather is described by a probability distribution. When these probability distributions are restricted to a…
The work concerns invariant measures for multivalued McKean-Vlasov stochastic differential equations. First of all, we prove the exponential ergodicity of these equations. Then for a sequence of these equations, when their coefficients…
Motivated by collaborative reinforcement learning (RL) and optimization with time-correlated data, we study a generic federated stochastic approximation problem involving $M$ agents, where each agent is characterized by an agent-specific…
The embedding problem for Markov chains is a famous problem in probability theory and only partial results are available up till now. In this paper, we propose a variant of the embedding problem called the reversible embedding problem which…
We consider the problem of fitting variational posterior approximations using stochastic optimization methods. The performance of these approximations depends on (1) how well the variational family matches the true posterior…
The study of stochastic variational principles involves the problem of constructing fixed-endpoint and adapted variations of semimartingales. We provide a detailed construction of variations of semimartingales that are not only fixed at…
The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…
Stochastic approximation is a powerful class of algorithms with celebrated success. However, a large body of previous analysis focuses on stochastic approximations driven by contractive operators, which is not applicable in some important…