Related papers: Saddlepoint approximation for Student's t-statisti…
It is known that step size adaptive evolution strategies (ES) do not converge (prematurely) to regular points of continuously differentiable objective functions. Among critical points, convergence to minima is desired, and convergence to…
In this paper, we focus on solving a class of constrained non-convex non-concave saddle point problems in a decentralized manner by a group of nodes in a network. Specifically, we assume that each node has access to a summand of a global…
Exponential tail bounds for sums play an important role in statistics, but the example of the $t$-statistic shows that the exponential tail decay may be lost when population parameters need to be estimated from the data. However, it turns…
We consider saddle point problems which objective functions are the average of $n$ strongly convex-concave individual components. Recently, researchers exploit variance reduction methods to solve such problems and achieve linear-convergence…
One of the ways to characterize a probability distribution is to show that it is moment-determinate, uniquely determined by knowing all its moments. The uniqueness, in the absolutely continuous case, depends entirely on the behaviour of the…
Superstatistics are superpositions of different statistics relevant for driven nonequilibrium systems with spatiotemporal inhomogeneities of an intensive variable (e.g., the inverse temperature). They contain Tsallis statistics as a special…
We show a statistical version of Taylor's theorem and apply this result to non-parametric density estimation from truncated samples, which is a classical challenge in Statistics \cite{woodroofe1985estimating, stute1993almost}. The…
We consider the problem of convergence to a saddle point of a concave-convex function via gradient dynamics. Since first introduced by Arrow, Hurwicz and Uzawa in [1] such dynamics have been extensively used in diverse areas, there are,…
We consider the problem of approximating the moment generating function (MGF) of a truncated random variable in terms of the MGF of the underlying (i.e., untruncated) random variable. The purpose of approximating the MGF is to enable the…
In this paper, we give a sharp analysis for Stochastic Gradient Descent (SGD) and prove that SGD is able to efficiently escape from saddle points and find an $(\epsilon, O(\epsilon^{0.5}))$-approximate second-order stationary point in…
This paper focuses on stochastic saddle point problems with decision-dependent distributions. These are problems whose objective is the expected value of a stochastic payoff function and whose data distribution drifts in response to…
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such…
The note considers normalized gradient descent (NGD), a natural modification of classical gradient descent (GD) in optimization problems. A serious shortcoming of GD in non-convex problems is that GD may take arbitrarily long to escape from…
Saddle point approximations, extremely important in a wide variety of physical contexts, require the analytical continuation of canonically conjugate quantities to complex variables in quantum mechanics. An important component of this…
This paper shows that a perturbed form of gradient descent converges to a second-order stationary point in a number iterations which depends only poly-logarithmically on dimension (i.e., it is almost "dimension-free"). The convergence rate…
In this paper we derive non-classical Tauberian asymptotic at infinity for the tail, the density and the derivatives thereof of a large class of exponential functionals of subordinators. More precisely, we consider the case when the L\'evy…
The class of dispersion models introduced by J{\o}rgensen (1997b) covers many known distributions such as the normal, Student t, gamma, inverse Gaussian, hyperbola, von-Mises, among others. We study the small dispersion asymptotic…
``Localization'' has proven to be a valuable tool in the Statistical Learning literature as it allows sharp risk bounds in terms of the problem geometry. Localized bounds seem to be much less exploited in the Stochastic Optimization…
Assuming X is a random vector and A a non-invertible matrix, one sometimes need to perform inference while only having access to samples of Y = AX. The corresponding likelihood is typically intractable. One may still be able to perform…
We investigate the uniform convergence of subdifferential mappings from empirical risk to population risk in nonsmooth, nonconvex stochastic optimization. This question is key to understanding how empirical stationary points approximate…