Related papers: Some Limit Properties of Markov Chains Induced by …
Methods for proving functional limit laws are developed for sequences of stochastic processes which allow a recursive distributional decomposition either in time or space. Our approach is an extension of the so-called contraction method to…
We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…
We present an algorithm that can efficiently compute a broad class of inferences for discrete-time imprecise Markov chains, a generalised type of Markov chains that allows one to take into account partially specified probabilities and other…
We consider an infinite-dimensional stochastic clustering model on $\mathbb{R}$. In discrete time, each point of a unit-intensity simple point process moves halfway toward either of its left or right neighbors, chosen uniformly at random.…
In the continuity of a recent paper ([6]), dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact…
The asymptotic behavior of stochastic gradient algorithms is studied. Relying on results from differential geometry (Lojasiewicz gradient inequality), the single limit-point convergence of the algorithm iterates is demonstrated and…
In this paper we propose a data-driven distributionally robust Model Predictive Control framework for constrained stochastic systems with unbounded additive disturbances. Recursive feasibility is ensured by optimizing over an linearly…
For obtaining optimal first-order convergence guarantee for stochastic optimization, it is necessary to use a recurrent data sampling algorithm that samples every data point with sufficient frequency. Most commonly used data sampling…
The asymptotic behavior of the stochastic gradient algorithm with a biased gradient estimator is analyzed. Relying on arguments based on the dynamic system theory (chain-recurrence) and the differential geometry (Yomdin theorem and…
Filtering---estimating the state of a partially observable Markov process from a sequence of observations---is one of the most widely studied problems in control theory, AI, and computational statistics. Exact computation of the posterior…
An algorithm is proposed, analyzed, and tested experimentally for solving stochastic optimization problems in which the decision variables are constrained to satisfy equations defined by deterministic, smooth, and nonlinear functions. It is…
We study the general approach to accelerating the convergence of the most widely used solution method of Markov decision processes with the total expected discounted reward. Inspired by the monotone behavior of the contraction mappings in…
The dynamics in games involving multiple players, who adaptively learn from their past experience, is not yet well understood. We analyzed a class of stochastic games with Markov strategies in which players choose their actions…
A multiplicative relative value iteration algorithm for solving the dynamic programming equation for the risk-sensitive control problem is studied for discrete time controlled Markov chains with a compact Polish state space, and controlled…
We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and…
A stochastic incremental subgradient algorithm for the minimization of a sum of convex functions is introduced. The method sequentially uses partial subgradient information and the sequence of partial subgradients is determined by a general…
Most existing literature on supervised machine learning assumes that the training dataset is drawn from an i.i.d. sample. However, many real-world problems exhibit temporal dependence and strong correlations between the marginal…
In this paper we study the effect of stochastic errors on two constrained incremental sub-gradient algorithms. We view the incremental sub-gradient algorithms as decentralized network optimization algorithms as applied to minimize a sum of…
The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…
In this paper, we consider a class of stochastic optimal control problems with risk constraints that are expressed as bounded probabilities of failure for particular initial states. We present here a martingale approach that diffuses a risk…