Related papers: Sharp Ascent--Descent Spectral Stability under Str…
Many problems in machine learning can be formulated as optimizing a convex functional over a vector space of measures. This paper studies the convergence of the mirror descent algorithm in this infinite-dimensional setting. Defining Bregman…
The stability of spike deconvolution, which aims at recovering point sources from their convolution with a point spread function (PSF), is known to be related to the separation between those sources. When the observations are noisy, it is…
In this paper, we introduce a new approach to proving the convergence of the Stochastic Approximation (SA) and the Stochastic Gradient Descent (SGD) algorithms. The new approach is based on a concept called GSLLN (Generalized Strong Law of…
In this paper Estimation of Signal Parameters via Rotational Invariance Techniques (ESPRIT) is developed for spectral estimation with single-snapshot measurement. Stability and resolution analysis with performance guarantee for…
Compressive sensing (CS) is well-known for its unique functionalities of sensing, compressing, and security (i.e. CS measurements are equally important). However, there is a tradeoff. Improving sensing and compressing efficiency with prior…
This paper revisits the convergence of Stochastic Mirror Descent (SMD) in the contemporary nonconvex optimization setting. Existing results for batch-free nonconvex SMD restrict the choice of the distance generating function (DGF) to be…
We provide the exact time-dependent cosmological solutions in the Randall-Sundrum (RS) setup with bulk matter, which may be smoothly connected to the static RS metric. In the static limit of the extra dimension, the solutions are reduced to…
We study the statistical properties of the dynamic trajectory of stochastic gradient descent (SGD). We approximate the mini-batch SGD and the momentum SGD as stochastic differential equations (SDEs). We exploit the continuous formulation of…
We propose a quantitative direct method to prove the local stability of a stationary solution for a rough differential equation and its regular discretization scheme. Using Doss-Sussmann technique and stopping time analysis, we provide…
SVRG and its variants are among the state of art optimization algorithms for large scale machine learning problems. It is well known that SVRG converges linearly when the objective function is strongly convex. However this setup can be…
Owing to its simplicity and efficiency, the Sherman-Morrison (SM) formula has seen widespread use across various scientific and engineering applications for solving rank-one perturbed linear systems of the form $(A+uv^T)x = b$. Although the…
Work of D. Stern and Bray-Kazaras-Khuri-Stern provide differential-geometric identities which relate the scalar curvature of Riemannian 3-manifolds to global invariants in terms of harmonic functions. These quantitative formulas are useful…
Recent work suggests that (stochastic) gradient descent self-organizes near an instability boundary, shaping both optimization and the solutions found. Momentum and mini-batch gradients are widely used in practical deep learning…
In this note, we clarify the stability of the large-N functional RG fixed points of the order/disorder transition in the random-field (RF) and random-anisotropy (RA) O(N) models. We carefully distinguish between infinite N, and large but…
We show non-linear stability and instability results in spherical symmetry for the interior of a charged black hole -approaching a sub-extremal Reissner-Nordstr\"om background fast enough at infinity- in presence of a massive and charged…
The robust statistical description of dynamical systems under perturbations is a central problem in ergodic theory. In this paper, we investigate the statistical properties of skew-product maps driven by a subshift of finite type with…
Continuing a line of investigation initiated in [11] exploring the connections between Jost and Evans functions and (modified) Fredholm determinants of Birman-Schwinger type integral operators, we here examine the stability index, or sign…
We prove explicit bounds on the exponential rate of convergence for the momentum stochastic gradient descent scheme (MSGD) for arbitrary, fixed hyperparameters (learning rate, friction parameter) and its continuous-in-time counterpart in…
In this work we explore the fidelity of numerical approximations to the analytic spectra of hyperbolic partial differential equation systems with variable coefficients. We are particularly interested in the ability of discrete methods to…
The stochastic proximal point (SPP) methods have gained recent attention for stochastic optimization, with strong convergence guarantees and superior robustness to the classic stochastic gradient descent (SGD) methods showcased at little to…