Related papers: NS-RGS: Newton-Schulz based Riemannian gradient me…

Neumann series underlie both Krylov methods and algebraic multigrid smoothers. A low-synch modified Gram-Schmidt (MGS)-GMRES algorithm is described that employs a Neumann series to accelerate the projection step. A corollary to the backward…

This thesis is about new methods of achieving RG transformations, in both a continuum spacetime background and on a lattice discretization thereof. The subject is explored from the point of view of euclidean quantum field theory. As a…

Numerical renormalization group (NRG) calculations of quantum impurity models, based on a logarithmic discretization in energy of electronic or bosonic Hamiltonians, provide a powerful tool to describe physics involving widely separated…

Stochastic gradient descent~(SGD) and its variants have been the dominating optimization methods in machine learning. Compared to SGD with small-batch training, SGD with large-batch training can better utilize the computational power of…

We provide the first theoretical analysis on the convergence rate of the asynchronous stochastic variance reduced gradient (SVRG) descent algorithm on non-convex optimization. Recent studies have shown that the asynchronous stochastic…

This paper considers a class of constrained stochastic composite optimization problems whose objective function is given by the summation of a differentiable (possibly nonconvex) component, together with a certain non-differentiable (but…

Nonequilibrium reaction networks (NRNs) underlie most biological functions. Despite their diverse dynamic properties, NRNs share the signature characteristics of persistent probability fluxes and continuous energy dissipation even in the…

We consider the problem of matrix completion with graphs as side information depicting the interrelations between variables. The key challenge lies in leveraging the similarity structure of the graph to enhance matrix recovery. Existing…

3D Gaussian Splatting (3DGS) has emerged as a promising approach for CT reconstruction. However, existing methods rely on the average gradient magnitude of points within the view, often leading to severe needle-like artifacts under…

The time-dependent numerical renormalization group (td-NRG) [Anders et al. Phys. Rev. Lett. {\bf 95}, 196801 (2006)] offers the prospect of investigating in a non-perturbative manner the time-dependence of local observables of interacting…

In this paper, based on the limited memory techniques and subspace minimization conjugate gradient (SMCG) methods, a regularized limited memory subspace minimization conjugate gradient method is proposed, which contains two types of…

This paper studies distributed nonconvex optimization problems with stochastic gradients for a multi-agent system, in which each agent aims to minimize the sum of all agents' cost functions by using local compressed information exchange. We…

In this paper we study the problem of minimizing the average of a large number ($n$) of smooth convex loss functions. We propose a new method, S2GD (Semi-Stochastic Gradient Descent), which runs for one or several epochs in each of which a…

In this paper we suggest a method by which reference broadcast synchronization (RBS), and other methods of estimating clock values, can be incorporated in standard clock synchronization algorithms to improve synchronization quality. We…

Finding an $\epsilon$-stationary point of a nonconvex function with a Lipschitz continuous Hessian is a central problem in optimization. Regularized Newton methods are a classical tool and have been studied extensively, yet they still face…

This paper presents several new algorithms for the regularized reconstruction of a surface from its measured gradient field. By taking a matrix-algebraic approach, we establish general framework for the regularized reconstruction problem…

Recovering an unknown signal from quadratic measurements has gained popularity due to its wide range of applications, including phase retrieval, fusion frame phase retrieval, and positive operator-valued measures. In this paper, we employ a…

Large-scale non-convex sparsity-constrained problems have recently gained extensive attention. Most existing deterministic optimization methods (e.g., GraSP) are not suitable for large-scale and high-dimensional problems, and thus…

Nonlinear conjugate gradient (NLCG) based optimizers have shown superior loss convergence properties compared to gradient descent based optimizers for traditional optimization problems. However, in Deep Neural Network (DNN) training, the…

A fast algorithm for solving the under-determined 3-D linear gravity inverse problem based on the randomized singular value decomposition (RSVD) is developed. The algorithm combines an iteratively reweighted approach for $L_1$-norm…