相关论文: Model Hessian for accelerating first-principles st…

When training large models, such as neural networks, the full derivatives of order 2 and beyond are usually inaccessible, due to their computational cost. Therefore, among the second-order optimization methods, it is common to bypass the…

Stochastic neighbor embedding (SNE) and related nonlinear manifold learning algorithms achieve high-quality low-dimensional representations of similarity data, but are notoriously slow to train. We propose a generic formulation of embedding…

Hessian-free (HF) optimization has been shown to effectively train deep autoencoders (Martens, 2010). In this paper, we aim to accelerate HF training of autoencoders by reducing the amount of data used in training. HF utilizes the conjugate…

In a Hilbert setting, we develop fast methods for convex unconstrained optimization. We rely on the asymptotic behavior of an inertial system combining geometric damping with temporal scaling. The convex function to minimize enters the…

We present a unified convergence analysis for first order convex optimization methods using the concept of strong Lyapunov conditions. Combining this with suitable time scaling factors, we are able to handle both convex and strong convex…

We introduce a new restarting scheme for a continuous inertial dynamics with Hessian driven-damping, and establish a linear convergence rate for the function values along the restarted trajectories. The proposed routine is implemented…

We provide an exact analysis of a class of randomized algorithms for solving overdetermined least-squares problems. We consider first-order methods, where the gradients are pre-conditioned by an approximation of the Hessian, based on a…

In this work we develop a Hessian-based sampling method for the construction of goal-oriented reduced order models with high-dimensional parameter inputs. Model reduction is known very challenging for high-dimensional parametric problems…

We derive and analyze a novel approach for modeling and computing the mechanical relaxation of incommensurate 2D heterostructures. Our approach parametrizes the relaxation pattern by the compact local configuration space rather than real…

We propose a family of optimization methods that achieve linear convergence using first-order gradient information and constant step sizes on a class of convex functions much larger than the smooth and strongly convex ones. This larger…

Stochastic second-order methods achieve fast local convergence in strongly convex optimization by using noisy Hessian estimates to precondition the gradient. However, these methods typically reach superlinear convergence only when the…

A fast and reliable geometry optimization algorithm is presented that optimizes atomic positions and lattice vectors simultaneously. Using a series of benchmarks, it is shown that the method presented in this paper outperforms in most cases…

This paper presents an efficient algorithm for the approximation of the rank-one convex hull in the context of nonlinear solid mechanics. It is based on hierarchical rank-one sequences and simultaneously provides first and second derivative…

Pruning neural networks reduces inference time and memory costs. On standard hardware, these benefits will be especially prominent if coarse-grained structures, like feature maps, are pruned. We devise two novel saliency-based methods for…

We want to propose a new discretization ansatz for the second order Hessian complex exploiting benefits of isogeometric analysis, namely the possibility of high-order convergence and smoothness of test functions. Although our approach is…

This paper proposes a new approach for the calibration of material parameters in local elastoplastic constitutive models. The calibration is posed as a constrained optimization problem, where the constitutive model evolution equations for a…

In this work, we develop new optimization algorithms that use approximate second-order information combined with the gradient regularization technique to achieve fast global convergence rates for both convex and non-convex objectives. The…

Joint diagonalization, the process of finding a shared set of approximate eigenvectors for a collection of matrices, arises in diverse applications such as multidimensional harmonic analysis or quantum information theory. This task is…

Efficient Boltzmann-sampling using first-principles methods is challenging for extended systems due to the steep scaling of electronic structure methods with the system size. Stochastic approaches provide a gentler system-size dependency at…

We study accelerated Krasnoselskii-Mann-type methods with preconditioners in both continuous and discrete time. From a continuous-time model, we derive a generalized fast Krasnoselskii-Mann method, providing a new yet simple proof of…