Related papers: On Hessian limit directions along non-oscillating …
We investigate the boundedness of the $H^\infty$-calculus by estimating the bound $b(\varepsilon)$ of the mapping $H^{\infty}\rightarrow \mathcal{B}(X)$: $f\mapsto f(A)T(\varepsilon)$ for $\varepsilon$ near zero. Here, $-A$ generates the…
It is well known that the non-spiraling leaves of real analytic foliations of codimension 1 all belong to the same o-minimal structure. Naturally, the question arises if the same statement is true for non-oscillating trajectories of real…
R. Thom's gradient conjecture states that if a gradient flow of an analytic function converges to a limit, it does so along a unique limiting direction. In this paper, we extend and settle this conjecture in the context of infinite…
The statistical properties of trajectories of eigenvalues of Gaussian complex matrices whose Hermitian condition is progressively broken are investigated. It is shown how the ordering on the real axis of the real eigenvalues is reflected in…
We consider the asymptotics of the difference between the empirical measures of the $\beta$-Hermite tridiagonal matrix and its minor. We prove that this difference has a deterministic limit and Gaussian fluctuations. Through a…
A characterization of the essential spectrum $\sigma_{\text ess}$ of Schr\"odinger operators on infinite graphs is derived involving the concept of $\mathcal{R}$-limits. This concept, which was introduced previously for operators on…
We develop a new algorithm for non-convex stochastic optimization that finds an $\epsilon$-critical point in the optimal $O(\epsilon^{-3})$ stochastic gradient and Hessian-vector product computations. Our algorithm uses Hessian-vector…
We consider stochastic gradient descents on the space of large symmetric matrices of suitable functions that are invariant under permuting the rows and columns using the same permutation. We establish deterministic limits of these random…
In this paper, we aim to study non-convex minimization problems via second-order (in-time) dynamics, including a non-vanishing viscous damping and a geometric Hessian-driven damping. Second-order systems that only rely on a viscous damping…
We prove sharp analytic regularity and decay at infinity of solutions of variable coefficients nonlinear harmonic oscillators. Namely, we show holomorphic extension to a sector in the complex domain, with a corresponding Gaussian decay,…
In this paper, we study the asymptotic behavior of continuous- and discrete-time gradient flows of a ``lower-unbounded" convex function $f$ on a Hadamard manifold $M$, particularly, their convergence properties to the boundary $M^{\infty}$…
The Hessian of neural networks can be decomposed into a sum of two matrices: (i) the positive semidefinite generalized Gauss-Newton matrix G, and (ii) the matrix H containing negative eigenvalues. We observe that for wider networks,…
Stochastic gradients for deep neural networks exhibit strong correlations along the optimization trajectory, and are often aligned with a small set of Hessian eigenvectors associated with outlier eigenvalues. Recent work shows that…
We investigate the location of the maximal gradient of the torsion function on certain non-symmetric planar domains. First, by establishing uniform estimates for convex narrow domains, we show that as a planar domain bounded by two graphs…
In this paper we study integer multiplicity rectifiable currents carried by the subgradient (subdifferential) graphs of semi-convex functions on a $n$-dimensional convex domain, and show a weak continuity theorem with respect to pointwise…
We study the asymptotic shape of the trajectory of the stochastic gradient descent algorithm applied to a convex objective function. Under mild regularity assumptions, we prove a functional central limit theorem for the properly rescaled…
We consider random resistor networks with nodes given by a point process on $\mathbb{R}^d$ and with random conductances. The length range of the electrical filaments can be unbounded. We assume that the randomness is stationary and ergodic…
We show that the derivatives in the sense of Fr\'echet and G\^ateaux can be viewed as derivatives oriented towards a star convex set with the origin as center. The resulting oriented differential calculus extends the mean value theorem, the…
Cohen et al. (2021) empirically study the evolution of the largest eigenvalue of the loss Hessian, also known as sharpness, along the gradient descent (GD) trajectory and observe the Edge of Stability (EoS) phenomenon. The sharpness…
This paper develops a general approach to characterize the long-time trajectory behavior of nonconvex gradient descent in generalized single-index models in the large aspect ratio regime. In this regime, we show that for each iteration the…