Related papers: A force-based gradient descent method for $\mathit…
We propose regularizing the empirical loss for semi-supervised learning by acting on both the input (data) space, and the weight (parameter) space. We show that the two are not equivalent, and in fact are complementary, one affecting the…
A set of accelerated first order algorithms with memory are proposed for minimising strongly convex functions. The algorithms are differentiated by their use of the iterate history for the gradient step. The increased convergence rate of…
For the problem of reconstructing a low-rank matrix from a few linear measurements, two classes of algorithms have been widely studied in the literature: convex approaches based on nuclear norm minimization, and non-convex approaches that…
This paper proposes a new easy-to-implement parameter-free gradient-based optimizer: DoWG (Distance over Weighted Gradients). We prove that DoWG is efficient -- matching the convergence rate of optimally tuned gradient descent in convex…
Based on multiple simulation trajectories, which started from dispersively selected initial conformations, the weighted ensemble dynamics method is designed to robustly and systematically explore the hierarchical structure of complex…
This is the first in a series of papers which deal with the development of novel methods for solving a system of linear algebraic equations with a time complexity lower than existing algorithms. The NxN system of linear equations, Ax = b,…
Upper Confidence Bound (UCB) is arguably the most commonly used method for linear multi-arm bandit problems. While conceptually and computationally simple, this method highly relies on the confidence bounds, failing to strike the optimal…
This work analyzes the solution trajectory of gradient-based algorithms via a novel basis function decomposition. We show that, although solution trajectories of gradient-based algorithms may vary depending on the learning task, they behave…
There are many challenges in bimanual assembly, including high-level sequencing, multi-robot coordination, and low-level, contact-rich operations such as component mating. Task and motion planning (TAMP) methods, while effective in this…
It is widely accepted that the stepsize is of great significance to gradient method. Two efficient gradient methods with approximately optimal stepsizes mainly based on regularization models are proposed for unconstrained optimization. More…
Variational Quantum Algorithms have emerged as a leading paradigm for near-term quantum computation. In such algorithms, a parameterized quantum circuit is controlled via a classical optimization method that seeks to minimize a…
Recent studies have shown that fractional calculus is an effective alternative mathematical tool in various scientific fields. However, some investigations indicate that results established in differential and integral calculus do not…
Sampling methods (e.g., node-wise, layer-wise, or subgraph) has become an indispensable strategy to speed up training large-scale Graph Neural Networks (GNNs). However, existing sampling methods are mostly based on the graph structural…
Two major momentum-based techniques that have achieved tremendous success in optimization are Polyak's heavy ball method and Nesterov's accelerated gradient. A crucial step in all momentum-based methods is the choice of the momentum…
Variational quantum algorithms are viewed as promising candidates for demonstrating quantum advantage on near-term devices. These approaches typically involve the training of parameterized quantum circuits through a classical optimization…
The GRadient Ascent Pulse Engineering (GRAPE) method is widely used for optimization in quantum control. GRAPE is gradient search method based on exact expressions for gradient of the control objective. It has been applied to coherently…
Finding the stationary states of a free energy functional is an important problem in phase field crystal (PFC) models. Many efforts have been devoted for designing numerical schemes with energy dissipation and mass conservation properties.…
We consider the framework of methods for unconstrained minimization that are, in each iteration, restricted to a model that is only a valid approximation to the objective function on some affine subspace containing an incumbent point. These…
In recent years, reduced basis methods (RBMs) have been adapted to the many-body eigenvalue problem and they have been used, largely in nuclear physics, as fast emulators able to bypass expensive direct computations while still providing…
Gradient descent method, as one of the major methods in numerical optimization, is the key ingredient in many machine learning algorithms. As one of the most fundamental way to solve the optimization problems, it promises the function value…