Related papers: Asymptotic normality and optimality in nonsmooth s…

Recently, a new local optimality concept for minimax problems, termed calm local minimax points, has been introduced. In this paper, we extend this concept to a general class of nonsmooth, nonconvex nonconcave minimax problems with coupled…

Asymptotic statistical theory for estimating functions is reviewed in a generality suitable for stochastic processes. Conditions concerning existence of a consistent estimator, uniqueness, rate of convergence, and the asymptotic…

Stochastic gradient descent (SGD) has been studied extensively over the past decades due to its simplicity and broad applicability in machine learning. In this work, we analyze the local behavior of gradient descent and stochastic gradient…

In this paper we propose a new method to stabilise non-symmetric indefinite problems. The idea is to solve a forward and an adjoint problem simultaneously using a suitable stabilised finite element method. Both stabilisation of the element…

Large-scale nonconvex optimization problems are ubiquitous in modern machine learning, and among practitioners interested in solving them, Stochastic Gradient Descent (SGD) reigns supreme. We revisit the analysis of SGD in the nonconvex…

In this paper, we investigate stochastic heat equation with sublinear diffusion coefficients. By assuming certain concavity of the diffusion coefficient, we establish non-trivial moment upper bounds and almost sure spatial asymptotic…

We study a nonlocal approximation of the Fokker-Planck equation in which we can estimate the speed of convergence to equilibrium in a way which does not degenerate as we approach the local limit of the equation. This uniform estimate cannot…

Adaptive gradient methods, such as AdaGrad, have become fundamental tools in deep learning. Despite their widespread use, the asymptotic convergence of AdaGrad remains poorly understood in non-convex scenarios. In this work, we present the…

We consider a family of algorithms that successively sample and minimize simple stochastic models of the objective function. We show that under reasonable conditions on approximation quality and regularity of the models, any such algorithm…

Optimization algorithms can see their local convergence rates deteriorate when the Hessian at the optimum is singular. These singularities are inescapable when the optima are non-isolated. Yet, under the right circumstances, several…

We consider an extension of the Newton-MR algorithm for nonconvex unconstrained optimization to the settings where Hessian information is approximated. Under a particular noise model on the Hessian matrix, we investigate the iteration and…

In this paper, we study the asymptotic behavior of a semi-linear slow-fast stochastic partial differential equation with singular coefficients. Using the Poisson equation in Hilbert space, we first establish the strong convergence in the…

Sharp asymptotic lower bounds of the expected quadratic variation of discretization error in stochastic integration are given. The theory relies on inequalities for the kurtosis and skewness of a general random variable which are themselves…

We examine a wide class of stochastic approximation algorithms for solving (stochastic) nonlinear problems on Riemannian manifolds. Such algorithms arise naturally in the study of Riemannian optimization, game theory and optimal transport,…

An algorithm is proposed, analyzed, and tested experimentally for solving stochastic optimization problems in which the decision variables are constrained to satisfy equations defined by deterministic, smooth, and nonlinear functions. It is…

Static spherically symmetric solutions to the Einstein-Euler equations with prescribed central densities are known to exist, be unique and smooth for reasonable equations of state. Some criteria are also available to decide whether…

This paper studies a stochastic algorithm for linearly constrained nonconvex optimization, where the objective function is smooth but only unbiased stochastic gradients with bounded variance are available. We propose a momentum-based…

M-estimation, aka empirical risk minimization, is at the heart of statistics and machine learning: Classification, regression, location estimation, etc. Asymptotic theory is well understood when the loss satisfies some smoothness…

A large number of problems in optimization, machine learning, signal processing can be effectively addressed by suitable semidefinite programming (SDP) relaxations. Unfortunately, generic SDP solvers hardly scale beyond instances with a few…

In a normed space setting, this paper studies the conditions under which the projected solutions to a quasi equilibrium problem with non-self constraint map exist. Our approach is based on an iterative algorithm which gives rise to a…