Related papers: Gradient Boosts the Approximate Vanishing Ideal

We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of…

Gradient-based algorithms, popular strategies to optimization problems, are essential for many modern machine-learning techniques. Theoretically, extreme points of certain cost functions can be found iteratively along the directions of the…

Universal methods for optimization are designed to achieve theoretically optimal convergence rates without any prior knowledge of the problem's regularity parameters or the accurarcy of the gradient oracle employed by the optimizer. In this…

Softmax is widely used in neural networks for multiclass classification, gate structure and attention mechanisms. The statistical assumption that the input is normal distributed supports the gradient stability of Softmax. However, when used…

We investigate the convergence properties of a class of iterative algorithms designed to minimize a potentially non-smooth and noisy objective function, which may be algebraically intractable and whose values may be obtained as the output…

It is well known that for gradient systems in Euclidean space or on a Riemannian manifold, the energy decreases monotonically along solutions. In this letter we derive and analyse functionally fitted energy-diminishing methods to preserve…

In this paper we introduce a working generalization of the theory of Gr\"obner bases for algebras of partial difference polynomials with constant coefficients. One obtains symbolic (formal) computation for systems of linear or non-linear…

Deep feedforward and recurrent networks have achieved impressive results in many perception and language processing applications. This success is partially attributed to architectural innovations such as convolutional and long short-term…

We study the ideal generated by polynomials vanishing on a semialgebraic set and propose an algorithm to calculate the generators, which is based on some techniques of the cylindrical algebraic decomposition. By applying these, polynomial…

The rapid progress in machine learning in recent years has been based on a highly productive connection to gradient-based optimization. Further progress hinges in part on a shift in focus from pattern recognition to decision-making and…

Many applications in machine learning or signal processing involve nonsmooth optimization problems. This nonsmoothness brings a low-dimensional structure to the optimal solutions. In this paper, we propose a randomized proximal gradient…

In this paper, we generalize the notion of border bases of zero-dimensional polynomial ideals to the module setting. To this end, we introduce order modules as a generalization of order ideals and module border bases of submodules with…

This work studies minimization problems with zero-order noisy oracle information under the assumption that the objective function is highly smooth and possibly satisfies additional properties. We consider two kinds of zero-order projected…

We address composite optimization problems, which consist in minimizing the sum of a smooth and a merely lower semicontinuous function, without any convexity assumptions. Numerical solutions of these problems can be obtained by proximal…

Solving zero-dimensional polynomial systems using Gr\"obner bases is usually done by, first, computing a Gr\"obner basis for the degree reverse lexicographic order, and next computing the lexicographic Gr\"obner basis with a change of order…

Gradient descent method, as one of the major methods in numerical optimization, is the key ingredient in many machine learning algorithms. As one of the most fundamental way to solve the optimization problems, it promises the function value…

In this paper, a gradient-free distributed algorithm is introduced to solve a set constrained optimization problem under a directed communication network. Specifically, at each time-step, the agents locally compute a so-called…

The quotient bases for zero-dimensional ideals are often of interest in the investigation of multivariate polynomial interpolation, algebraic coding theory, and computational molecular biology, etc. In this paper, we discuss the properties…

In this article we introduce an algorithm for mitigating the adverse effects of noise on gradient descent in variational quantum algorithms. This is accomplished by computing a {\emph{regularized}} local classical approximation to the…

In this paper we characterize, in algebraic and geometric terms, when a graded vanishing ideal is generated by binomials over any field K.