Related papers: Krylov Methods for Adjoint-Free Singular Vector Ba…
This paper proposes a provably convergent multiblock ADMM for nonconvex optimization with nonlinear dynamics constraints, overcoming the divergence issue in classical extensions. We consider a class of optimization problems that arise from…
Most current prevalent iterative methods can be classified into the so-called extended Krylov subspace methods, a class of iterative methods which do not fall into this category are also proposed in this paper. Comparing with traditional…
This work shows how the early stages of perturbation growth in a viscosity-stratified flow are different from those in a constant-viscosity flow, and how nonlinearity is a crucial ingredient. We derive the viscosity-varying adjoint…
First order optimization algorithms play a major role in large scale machine learning. A new class of methods, called adaptive algorithms, were recently introduced to adjust iteratively the learning rate for each coordinate. Despite great…
Ionides, King et al. (see e.g. Inference for nonlinear dynamical systems, PNAS 103) have recently introduced an original approach to perform maximum likelihood parameter estimation in state-space models which only requires being able to…
We propose a principled method for projecting an arbitrary square matrix to the non-convex set of asymptotically stable matrices. Leveraging ideas from large deviations theory, we show that this projection is optimal in an…
A variety of shooting methods for computing fully discrete time-periodic solutions of partial differential equations, including Newton-Krylov and optimization-based methods, are discussed and used to determine the periodic, compressible,…
We study the convergence of accelerated stochastic gradient descent for strongly convex objectives under the growth condition, which states that the variance of stochastic gradient is bounded by a multiplicative part that grows with the…
We propose a tridiagonalization approach for non-Hermitian random matrices and Hamiltonians using singular value decomposition (SVD). This technique leverages the real and non-negative nature of singular values, bypassing the complex…
We develop two new variants of alternating direction methods of multipliers (ADMM) and two parallel primal-dual decomposition algorithms to solve a wide range class of constrained convex optimization problems. Our approach relies on a novel…
In this paper we present an active-set method for the solution of $\ell_1$-regularized convex quadratic optimization problems. It is derived by combining a proximal method of multipliers (PMM) strategy with a standard semismooth Newton…
We propose an acceleration scheme for first-order methods (FOMs) for convex quadratic programs (QPs) that is analogous to Anderson acceleration and the Generalized Minimal Residual algorithm for linear systems. We motivate our proposed…
We present an implementation of the hybridization expansion impurity solver which employs sparse matrix exact-diagonalization techniques to compute the time evolution of the local Hamiltonian. This method avoids computationally expensive…
In this work we investigate stochastic non-convex optimization problems where the objective is an expectation over smooth loss functions, and the goal is to find an approximate stationary point. The most popular approach to handling such…
Krylov complexity is an important dynamical quantity with relevance to the study of operator growth and quantum chaos, and has recently been much studied for various time-independent systems. We initiate the study of K-complexity in…
We propose an improved scheme to do the time dependent variational principle (TDVP) in finite matrix product states (MPS) for two-dimensional systems or one-dimensional systems with long range interactions. We present a method to represent…
This paper presents a sufficient condition for stochastic gradients not to slow down the convergence of Nesterov's accelerated gradient method. The new condition has the strong-growth condition by Schmidt \& Roux as a special case, and it…
We present a dynamical system framework for understanding Nesterov's accelerated gradient method. In contrast to earlier work, our derivation does not rely on a vanishing step size argument. We show that Nesterov acceleration arises from…
Tensor train decomposition is a powerful tool for dealing with high-dimensional, large-scale tensor data, which is not suffering from the curse of dimensionality. To accelerate the calculation of the auxiliary unfolding matrix, some…
We propose a hybrid meta-learning framework for forecasting and anomaly detection in nonlinear dynamical systems characterized by nonstationary and stochastic behavior. The approach integrates a physics-inspired simulator that captures…