Related papers: NS-RGS: Newton-Schulz based Riemannian gradient me…

Quantum impurity problems can be solved using the numerical renormalization group (NRG), which involves discretizing the free conduction electron system and mapping to a `Wilson chain'. It was shown recently that Wilson chains for different…

We investigate the Randomized Stochastic Accelerated Gradient (RSAG) method, utilizing either constant or adaptive step sizes, for stochastic optimization problems with generalized smooth objective functions. Under relaxed affine variance…

Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions. The present paper proposes a Riemannian stochastic quasi-Newton algorithm with variance…

The renormalization group (RG) constitutes a fundamental framework in modern theoretical physics. It allows the study of many systems showing states with large-scale correlations and their classification in a relatively small set of…

We present an algorithm for minimizing a sum of functions that combines the computational efficiency of stochastic gradient descent (SGD) with the second order curvature information leveraged by quasi-Newton methods. We unify these…

Stochastic gradient descent (SGD) is a widely adopted iterative method for optimizing differentiable objective functions. In this paper, we propose and discuss a novel approach to scale up SGD in applications involving non-convex functions…

Optimization techniques are of great importance to effectively and efficiently train a deep neural network (DNN). It has been shown that using the first and second order statistics (e.g., mean and variance) to perform Z-score…

We study a generic problem of dissipative quantum mechanics, a small local quantum system with discrete states coupled in an arbitrary way (i.e. not necessarily linear) to several infinitely large particle or heat reservoirs. For both…

We propose a Randomised Subspace Gauss-Newton (R-SGN) algorithm for solving nonlinear least-squares optimization problems, that uses a sketched Jacobian of the residual in the variable domain and solves a reduced linear least-squares on…

The Numerical Renormalization Group method (NRG) has been developed by Wilson in the 1970's to investigate the Kondo problem. The NRG allows the non-perturbative calculation of static and dynamic properties for a variety of impurity models.…

Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite, number of loss functions. In this paper, we propose a novel Riemannian extension of the Euclidean stochastic variance…

In this paper, we propose a first second-order scheme based on arbitrary non-Euclidean norms, incorporated by Bregman distances. They are introduced directly in the Newton iterate with regularization parameter proportional to the square…

Optimization techniques in deep learning are predominantly led by first-order gradient methodologies, such as SGD. However, neural network training can greatly benefit from the rapid convergence characteristics of second-order optimization.…

We present a hybrid lattice Hamiltonian truncation method that integrates the numerical renormalization group (NRG) with a truncated lattice integrable spectrum. The technique is tailored for generic deformations of integrable lattice…

We describe novel subgradient methods for a broad class of matrix optimization problems involving nuclear norm regularization. Unlike existing approaches, our method executes very cheap iterations by combining low-rank stochastic…

Many geometric estimation problems take the form of synchronization over the special Euclidean group: estimate the values of a set of poses given noisy measurements of a subset of their pairwise relative transforms. This problem is…

In this contribution, we present a numerical analysis of the continuous stochastic gradient (CSG) method, including applications from topology optimization and convergence rates. In contrast to standard stochastic gradient optimization…

Under mild assumptions stochastic gradient methods asymptotically achieve an optimal rate of convergence if the arithmetic mean of all iterates is returned as an approximate optimal solution. However, in the absence of stochastic noise, the…

In this paper, we study the efficiency of a {\bf R}estarted {\bf S}ub{\bf G}radient (RSG) method that periodically restarts the standard subgradient method (SG). We show that, when applied to a broad class of convex optimization problems,…

We propose a modified form of a tensor renormalization group algorithm for evaluating partition functions of classical statistical mechanical models on 2D lattices. This algorithm coarse-grains only the rows and columns of the lattice…