Related papers: Gradient descent in hyperbolic space
Stochastic gradient methods have been a popular and powerful choice of optimization methods, aimed at minimizing functions. Their advantage lies in the fact that that one approximates the gradient as opposed to using the full Jacobian…
This paper develops a notion of geometric quantiles on Hadamard spaces, also known as global non-positive curvature spaces. After providing some definitions and basic properties, including scaled isometry equivariance and a necessary…
We consider a volume maximization program to construct hyperbolic structures on triangulated 3-manifolds, for which previous progress has lead to consider angle assignments which do not correspond to a hyperbolic metric on each simplex. We…
Hyperbolic space is a geometry that is known to be well-suited for representation learning of data with an underlying hierarchical structure. In this paper, we present a novel hyperbolic distribution called \textit{pseudo-hyperbolic…
Representing data in hyperbolic space can effectively capture latent hierarchical relationships. With the goal of enabling accurate classification of points in hyperbolic space while respecting their hyperbolic geometry, we introduce…
In the recent years, various gradient descent algorithms including the methods of gradient descent, gradient descent with momentum, adaptive gradient (AdaGrad), root-mean-square propagation (RMSProp) and adaptive moment estimation (Adam)…
We show that in a variety of large-scale deep learning scenarios the gradient dynamically converges to a very small subspace after a short period of training. The subspace is spanned by a few top eigenvectors of the Hessian (equal to the…
In this paper, we propose a geometric framework to analyze the convergence properties of gradient descent trajectories in the context of linear neural networks. We translate a well-known empirical observation of linear neural nets into a…
We prove a sharp area estimate for minimal submanifolds that pass through a prescribed point in a geodesic ball in hyperbolic space, in any dimension and codimension. In certain cases, we also prove the corresponding estimate in the sphere.…
Deep representation learning is a ubiquitous part of modern computer vision. While Euclidean space has been the de facto standard manifold for learning visual representations, hyperbolic space has recently gained rapid traction for learning…
Although Deep Learning (DL) has achieved success in complex Artificial Intelligence (AI) tasks, it suffers from various notorious problems (e.g., feature redundancy, and vanishing or exploding gradients), since updating parameters in…
Network data is ubiquitous in various scientific disciplines, including sociology, economics, and neuroscience. Latent space models are often employed in network data analysis, but the geometric effect of latent space curvature remains a…
We develop a new Riemannian descent algorithm that relies on momentum to improve over existing first-order methods for geodesically convex optimization. In contrast, accelerated convergence rates proved in prior work have only been shown to…
We investigate the problem of finding smooth hypersurfaces of constant mean curvature in hyperbolic space, which can be represented as radial graphs over a subdomain of the upper hemisphere. Our approach is variational and our main results…
We study the convergence rate of Bregman gradient methods for convex optimization in the space of measures on a $d$-dimensional manifold. Under basic regularity assumptions, we show that the suboptimality gap at iteration $k$ is in…
Hyperbolic spaces have recently gained momentum in the context of machine learning due to their high capacity and tree-likeliness properties. However, the representational power of hyperbolic geometry is not yet on par with Euclidean…
We establish pointwise ergodic theorems for a large class of natural averages on simple Lie groups of real-rank-one, going well beyond the radial case considered previously. The proof is based on a new approach to pointwise ergodic…
Higher-dimensional spaces are ubiquitous in applications of mathematics. Yet, as we live in a three-dimensional space, visualizing, say, a four-dimensional space is challenging. We introduce a novel method of interactive visualization of…
To compute the spatially distributed dielectric constant from the backscattering data, we study a coefficient inverse problem for a 1D hyperbolic equation. To solve the inverse problem, we establish a new version of Carleman estimate and…
Metric learning aims to learn a highly discriminative model encouraging the embeddings of similar classes to be close in the chosen metrics and pushed apart for dissimilar ones. The common recipe is to use an encoder to extract embeddings…