Related papers: Model Hessian for accelerating first-principles st…
First-order methods for solving convex optimization problems have been at the forefront of mathematical optimization in the last 20 years. The rapid development of this important class of algorithms is motivated by the success stories…
We propose a new first-order primal-dual optimization framework for a convex optimization template with broad applications. Our optimization algorithms feature optimal convergence guarantees under a variety of common structure assumptions…
Hidden Markov Models (HMMs) can be accurately approximated using co-occurrence frequencies of pairs and triples of observations by using a fast spectral method in contrast to the usual slow methods like EM or Gibbs sampling. We provide a…
A high-order accurate implicit-mesh discontinuous Galerkin framework for wave propagation in single-phase and bi-phase solids is presented. The framework belongs to the embedded-boundary techniques and its novelty regards the spatial…
Primal-Dual Hybrid Gradient (PDHG) and Alternating Direction Method of Multipliers (ADMM) are two widely-used first-order optimization methods. They reduce a difficult problem to simple subproblems, so they are easy to implement and have…
Iterative linear solvers have gained recent popularity due to their computational efficiency and low memory footprint for large-scale linear systems. The relaxation method, or Motzkin's method, can be viewed as an iterative method that…
We further research on the accelerated optimization phenomenon on Riemannian manifolds by introducing accelerated global first-order methods for the optimization of $L$-smooth and geodesically convex (g-convex) or $\mu$-strongly g-convex…
Reduced-order simulation is an emerging method for accelerating physical simulations with high DOFs, and recently developed neural-network-based methods with nonlinear subspaces have been proven effective in diverse applications as more…
We apply state-of-the-art tools in modern high-dimensional numerical linear algebra to approximate efficiently the spectrum of the Hessian of modern deepnets, with tens of millions of parameters, trained on real data. Our results…
This chapter provides a tutorial overview of first principles methods to describe the properties of matter at the ground state or equilibrium. It begins with a brief introduction to quantum and statistical mechanics for predicting the…
Motivated by robust matrix recovery problems such as Robust Principal Component Analysis, we consider a general optimization problem of minimizing a smooth and strongly convex loss function applied to the sum of two blocks of variables,…
In this article, we discuss the stability of soft quasicrystalline phases in a coupled-mode Swift-Hohenberg model for three-component systems, where the characteristic length scales are governed by the positive-definite gradient terms.…
Given an optimization problem, the Hessian matrix and its eigenspectrum can be used in many ways, ranging from designing more efficient second-order algorithms to performing model analysis and regression diagnostics. When nonlinear models…
We study the optimization of non-convex functions that are not necessarily smooth (gradient and/or Hessian are Lipschitz) using first order methods. Smoothness is a restrictive assumption in machine learning in both theory and practice,…
We study the robustness of accelerated first-order algorithms to stochastic uncertainties in gradient evaluation. Specifically, for unconstrained, smooth, strongly convex optimization problems, we examine the mean-squared error in the…
A few recent works explored incorporating geometric priors to regularize the optimization of Gaussian splatting, further improving its performance. However, those early studies mainly focused on the use of low-order geometric priors (e.g.,…
Molecular vibrations in solutions, especially OH stretching and bending in water, drive ultrafast energy relaxation and dephasing in chemical and biological systems. We present a machine learning approach for constructing system-bath models…
We are interested in a reduced order method for the efficient simulation of blood flow in arteries. The blood dynamics is modeled by means of the incompressible Navier-Stokes equations. Our algorithm is based on an approximated…
In this work, we revisit Transformer optimization through the lens of second-order geometry and establish a direct connection between architectural design, activation scale, the Hessian matrix, and the maximum tolerable learning rate. We…
Dense stellar systems such as globular clusters, galactic nuclei and nuclear star clusters are ideal loci to study stellar dynamics due to the very high densities reached, usually a million times higher than in the solar neighborhood; they…