Related papers: Stochastic approximation on non-compact measure sp…
In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…
We provide strong $L_p$-rates of approximation of nonsmooth integral-type functionals of Markov processes by integral sums. Our approach is, in a sense, process insensitive and is based on a modification of some well-developed estimates…
We develop a comprehensive theory for a general class of multi-parameter function spaces of Besov-Triebel-Lizorkin type, with a matrix weight. We prove the equivalence of different quasi-norms, the identification of function and sequence…
Motivated by applications in reinforcement learning (RL), we study a nonlinear stochastic approximation (SA) algorithm under Markovian noise, and establish its finite-sample convergence bounds under various stepsizes. Specifically, we show…
In this paper we present an inexact zeroth-order method suitable for the solution nonsmooth and nonconvex stochastic composite optimization problems, in which the objective is split into a real-valued Lipschitz continuous stochastic…
We prove the almost sure weak convergence of a stochastic proximal point method for minimizing a convex integral function in the general nonlinear context of complete geodesic metric spaces of nonpositive curvature (so-called Hadamard…
We prove a general quantitative theorem on the asymptotic behavior of stochastic quasi-Fej\'er monotone sequences in a broad metric context. Concretely, our result explicitly constructs a rate of convergence for such process, both in mean…
We propose and study a general quasi-interpolation framework for stochastic function approximation, which stems and draws motivation from convolution-type solutions for certain practical weighted variational problems. We obtain our…
In this work, we examine sampling problems with non-smooth potentials. We propose a novel Markov chain Monte Carlo algorithm for sampling from non-smooth potentials. We provide a non-asymptotical analysis of our algorithm and establish a…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
We propose a single time-scale stochastic subgradient method for constrained optimization of a composition of several nonsmooth and nonconvex functions. The functions are assumed to be locally Lipschitz and differentiable in a generalized…
A stochastic algorithm is proposed, finding some elements from the set of intrinsic $p$-mean(s) associated to a probability measure $\nu$ on a compact Riemannian manifold and to $p\in[1,\infty)$. It is fed sequentially with independent…
We introduce a new concept of dissipative measure-valued martingale solutions to the stochastic compressible Euler equations. These solutions are weak in the probabilistic sense i.e., the probability space and the driving Wiener process are…
We consider Markov decision processes (MDPs) with \omega-regular specifications given as parity objectives. We consider the problem of computing the set of almost-sure winning states from where the objective can be ensured with probability…
This article deals with stochastic processes endowed with the Markov (memoryless) property and evolving over general (uncountable) state spaces. The models further depend on a non-deterministic quantity in the form of a control input, which…
We study quasi-stationary distributions and quasi-limiting behavior of Markov chains in general reducible state spaces with absorption. We propose a set of assumptions dealing with particular situations where the state space can be…
We present an application of the theory of stochastic processes to model and categorize non-equilibrium physical phenomena. The concepts of uniformly continuous probability measures and modular evolution lead to a systematic hierarchical…
Sequential Monte Carlo Samplers are a class of stochastic algorithms for Monte Carlo integral estimation w.r.t. probability distributions, which combine elements of Markov chain Monte Carlo methods and importance sampling/resampling…
The paper suggests a generalization of the Sign-Perturbed Sums (SPS) finite sample system identification method for the identification of closed-loop observable stochastic linear systems in state-space form. The solution builds on the…
Stochastic convergence of discrete time Markov processes has been analysed based on a dual Lyapunov approach. Using some existing results on ergodic theory of Markov processes, it has been shown that existence of a properly subinvariant…