Related papers: When is the average number of saddle points typica…
We introduce the smoothed analysis of algorithms, which is a hybrid of the worst-case and average-case analysis of algorithms. In smoothed analysis, we measure the maximum over inputs of the expected performance of an algorithm under small…
Stationarity is a very general, qualitative assumption, that can be assessed on the basis of application specifics. It is thus a rather attractive assumption to base statistical analysis on, especially for problems for which less general…
A central challenge to using first-order methods for optimizing nonconvex problems is the presence of saddle points. First-order methods often get stuck at saddle points, greatly deteriorating their performance. Typically, to escape from…
We study the performance of stochastic first-order methods for finding saddle points of convex-concave functions. A notorious challenge faced by such methods is that the gradients can grow arbitrarily large during optimization, which may…
Average-case analysis computes the complexity of an algorithm averaged over all possible inputs. Compared to worst-case analysis, it is more representative of the typical behavior of an algorithm, but remains largely unexplored in…
We consider saddle point problems which objective functions are the average of $n$ strongly convex-concave individual components. Recently, researchers exploit variance reduction methods to solve such problems and achieve linear-convergence…
We study the problem of minimizing the average of a very large number of smooth functions, which is of key importance in training supervised learning models. One of the most celebrated methods in this context is the SAGA algorithm. Despite…
This paper investigates the supervised learning problem with observations drawn from certain general stationary stochastic processes. Here by \emph{general}, we mean that many stationary stochastic processes can be included. We show that…
We discuss various critical densities in sandpile models. The stationary density is the average expected height in the stationary state of a finite-volume model; the transition density is the critical point in the infinite-volume…
Hypergraphs are structures that can be decomposed or described; in other words they are recursively countable. Here, we get exact and asymptotic enumeration results on hypergraphs by means of exponential generating functions. The number of…
Most approximation methods in high dimensions exploit smoothness of the function being approximated. These methods provide poor convergence results for non-smooth functions with kinks. For example, such kinks can arise in the uncertainty…
Bauschke and Moursi have recently obtained results that implicitly contain the fact that the composition of finitely many averaged mappings on a Hilbert space that have approximate fixed points also has approximate fixed points and thus is…
The study of quenched random systems is facilitated by the idea that the ensemble averages describe the thermal averages for any specific realization of the couplings, provided the system is large enough. Careful examination suggests that…
We consider convex-concave saddle-point problems where the objective functions may be split in many components, and extend recent stochastic variance reduction methods (such as SVRG or SAGA) to provide the first large-scale linearly…
This paper studies the generalization bounds for the empirical saddle point (ESP) solution to stochastic saddle point (SSP) problems. For SSP with Lipschitz continuous and strongly convex-strongly concave objective functions, we establish…
We provide the first non-asymptotic analysis for finding stationary points of nonsmooth, nonconvex functions. In particular, we study the class of Hadamard semi-differentiable functions, perhaps the largest class of nonsmooth functions for…
Skewness measures can be used to measure the level of asymmetry of a distribution. Given the prevalence of statistical methods that assume underlying symmetry, and also the desire for symmetry in order to make meaningful judgements for…
Finding the mean of the total number of stationary points for N-dimensional random Gaussian landscapes can be reduced to averaging the absolute value of characteristic polynomial of the corresponding Hessian. First such a reduction is…
We analyze the landscape of general smooth Gaussian functions on the sphere in dimension $N$, when $N$ is large. We give an explicit formula for the asymptotic complexity of the mean number of critical points of finite and diverging index…
This paper establishes the functional average as an important estimand for causal inference. The significance of the estimand lies in its robustness against traditional issues of confounding. We prove that this robustness holds even when…