Related papers: Gradient descent in hyperbolic space

Microcanonical gradient descent is a sampling procedure for energy-based models allowing for efficient sampling of distributions in high dimension. It works by transporting samples from a high-entropy distribution, such as Gaussian white…

We study optimization over Riemannian embedded submanifolds, where the objective function is relatively smooth in the ambient Euclidean space. Such problems have broad applications but are still largely unexplored. We introduce two…

Bayesian experimental design (BED) is to answer the question that how to choose designs that maximize the information gathering. For implicit models, where the likelihood is intractable but sampling is possible, conventional BED methods…

Interpreting gradient methods as fixed-point iterations, we provide a detailed analysis of those methods for minimizing convex objective functions. Due to their conceptual and algorithmic simplicity, gradient methods are widely used in…

We study two procedures (reverse-mode and forward-mode) for computing the gradient of the validation error with respect to the hyperparameters of any iterative learning algorithm such as stochastic gradient descent. These procedures mirror…

The techniques and analysis presented in this thesis provide new methods to solve optimization problems posed on Riemannian manifolds. These methods are applied to the subspace tracking problem found in adaptive signal processing and…

We study expansions near the boundary of solutions to the Dirichlet problem for the constant mean curvature equation in the hyperbolic space. With a characterization of remainders of the expansion by multiple integrals, we establish optimal…

We propose a stochastic variance-reduced cubic regularized Newton algorithm to optimize the finite-sum problem over a Riemannian submanifold of the Euclidean space. The proposed algorithm requires a full gradient and Hessian update at the…

In this paper we continue our study of finding the curvature flow of complete hypersurfaces in hyperbolic space with a prescribed asymptotic boundary at infinity. Our main results are proved by deriving a priori global gradient estimates…

Problems of quadratic optimization in Hilbert space often arise when solving ill-posed problems for differential equations. In this case, the target value of the functional is known. In addition, the structure of the functional allows…

Inspired by a recent hyperbolic regularization of Burnett's hydrodynamic equations [A. Bobylev, J. Stat. Phys. 124, 371 (2006)], we introduce a method to derive hyperbolic equations of linear hydrodynamics to any desired accuracy in Knudsen…

Bilevel optimization is a central tool in machine learning for high-dimensional hyperparameter tuning. Its applications are vast; for instance, in imaging it can be used for learning data-adaptive regularizers and optimizing forward…

This paper considers optimization of smooth nonconvex functionals in smooth infinite dimensional spaces. A H\"older gradient descent algorithm is first proposed for finding approximate first-order points of regularized polynomial…

Asymptotic results for weighted floating bodies are established and used to obtain new proofs for the existence of floating areas on the sphere and in hyperbolic space and to establish the existence of floating areas in Hilbert geometries.…

The Riemannian manifold of curves with a Sobolev metric is an important and frequently studied model in the theory of shape spaces. Various numerical approaches have been proposed to compute geodesics, but so far elude a rigorous…

We adapt the Gradient Discretisation Method (GDM), originally designed for elliptic and parabolic partial differential equations, to the case of a linear scalar hyperbolic equations. This enables the simultaneous design and convergence…

Gradient-based iterative optimization methods are the workhorse of modern machine learning. They crucially rely on careful tuning of parameters like learning rate and momentum. However, one typically sets them using heuristic approaches…

Scientists and engineers rely on accurate mathematical models to quantify the objects of their studies, which are often high-dimensional. Unfortunately, high-dimensional models are inherently difficult, i.e. when observations are sparse or…

Computing the loss gradient via backpropagation consumes considerable energy during deep learning (DL) model training. In this paper, we propose a novel approach to efficiently compute DL models' gradients to mitigate the substantial energy…

The Heston model is a well-known two-dimensional financial model. Because the Heston model contains implicit parameters that cannot be determined directly from real market data, calibrating the parameters to real market data is challenging.…