Related papers: An abstract effective convergence theorem for stoc…
Nonconvex optimization is central to modern machine learning, but the general framework of nonconvex optimization yields weak convergence guarantees that are too pessimistic compared to practice. On the other hand, while convexity enables…
This work provides a novel convergence analysis for stochastic optimization in terms of stopping times, addressing the practical reality that algorithms are often terminated adaptively based on observed progress. Unlike prior approaches,…
In this paper, we consider a class of nonautonomous multi-scale stochastic partial differential equations with fully local monotone coefficients. By introducing the evolution system of measures for time-inhomogeneous Markov semigroups, we…
We provide quantitative and abstract strong convergence results for sequences from a compact metric space satisfying a certain form of \emph{generalized Fej\'er monotonicity} where (1) the metric can be replaced by a much more general type…
Two-time-scale stochastic approximation, a generalized version of the popular stochastic approximation, has found broad applications in many areas including stochastic control, optimization, and machine learning. Despite its popularity,…
In this paper, we propose and analyze a trust-region model-based algorithm for solving unconstrained stochastic optimization problems. Our framework utilizes random models of an objective function $f(x)$, obtained from stochastic…
In this paper we survey the almost sure central limit theorem and its functional form (quenched) for stationary and ergodic processes. For additive functionals of a stationary and ergodic Markov chain these theorems are known under the…
We study the asymptotic behaviour of the well-known Dykstra's algorithm through the lens of proof-theoretical techniques. We provide an elementary proof for the convergence of Dykstra's algorithm in which the standard argument is stripped…
In this article we introduce several kinds of easily implementable explicit schemes, which are amenable to Khasminski's techniques and are particularly suitable for highly nonlinear stochastic differential equations (SDEs). We show that…
Stochastic Gradient Descent (SGD) is widely used in machine learning research. Previous convergence analyses of SGD under the vanishing step-size setting typically require Robbins-Monro conditions. However, in practice, a wider variety of…
We study the almost sure convergence of the Stochastic Approximation algorithm to the fixed point $x^\star$ of a nonlinear operator under a negative drift condition and a general noise sequence with finite $p$-th moment for some $p > 1$.…
A finite dimensional abstract approximation and convergence theory is developed for estimation of the distribution of random parameters in infinite dimensional discrete time linear systems with dynamics described by regularly dissipative…
We study the performance of asymptotic and approximate consensus algorithms under harsh environmental conditions. The asymptotic consensus problem requires a set of agents to repeatedly set their outputs such that the outputs converge to a…
We study critera for a pair $ (\{ X_n \} $, $ \{ Y_n \}) $ of approximating processes which guarantee closeness of moments by generalizing known results for the special case that $ Y_n = Y $ for all $n$ and $ X_n $ converges to $Y$ in…
Many results related to quantitative problems in the metric theory of Diophantine approximation are asymptotic, such as the number of rational solutions to certain inequalities grows with the same rate almost everywhere modulo an asymptotic…
In this paper, we study the asymptotic behavior of a fully-coupled slow-fast McKean-Vlasov stochastic system. Using the non-linear Poisson equation on Wasserstein space, we first establish the strong convergence in the averaging principle…
We provide a quantitative version of a result due to Poffald and Reich on the asymptotic behavior of solutions of a second-order Cauchy problem generated by an accretive operator in the form of a rate of convergence. This quantitative…
This paper derives the rate of convergence and asymptotic distribution for a class of Kolmogorov-Smirnov style test statistics for conditional moment inequality models for parameters on the boundary of the identified set under general…
Motivated by applications arising from large scale optimization and machine learning, we consider stochastic quasi-Newton (SQN) methods for solving unconstrained convex optimization problems. The convergence analysis of the SQN methods,…
Martingale methods are used to study the almost everywhere convergence of general function series. Applications are given to ergodic series, which improves recent results of Fan \cite{FanETDS}, and to dilated series, including Davenport…