Related papers: Comments on "Generalization of the gradient method…
Recent work across many machine learning disciplines has highlighted that standard descent methods, even without explicit regularization, do not merely minimize the training error, but also exhibit an implicit bias. This bias is typically…
In this article we analyze a generalized trapezoidal rule for initial value problems with piecewise smooth right hand side \(F:\R^n\to\R^n\). When applied to such a problem the classical trapezoidal rule suffers from a loss of accuracy if…
The fractional quantization of singular systems with second order Lagrangian is examined. The fractional singular Lagrangian is presented. The equations of motion are written as total differential equations within fractional calculus. Also,…
Some divergent trigonometric integrals have appeared in standard tables for many years, listed as converging. We give a simple proof that these integrals diverge and trace their history. The original error was made when a (startlingly)…
We introduce a detailed analysis of the convergence of first-order methods with composite noise (sum of relative and absolute) in gradient for convex and smooth function minimization. This paper illustrates instances of practical problems…
This note provides a new approach to a result of Foregger and related earlier results by Keilson and Eberlein. Using quite different techniques, we prove a more general result from which the others follow easily. Finally, we argue that the…
We extend classical results on variational inequalities with convex sets with gradient constraint to a new class of fractional partial differential equations in a bounded domain with constraint on the distributional Riesz fractional…
In the paper Factorisation of division polynomials (H. Verdure, Proc. japan Academy, Ser A. 80 n. 5), Verdure gives the factorisation patterns of division polynomials of elliptic curves defined over a finite field. However, the result given…
This paper is devoted to the $L^p(\mathbb R)$ theory of the fractional Fourier transform (FRFT) for $1\le p < 2$. In view of the special structure of the FRFT, we study FRFT properties of $L^1$ functions, via the introduction of a suitable…
We point out a major flaw in the conformable calculus. We demonstrate why it fails at defining a fractional derivative and where exactly these tempting conformability properties come from.
Often in the analysis of first-order methods, assuming the existence of a quadratic growth bound (a generalization of strong convexity) facilitates much stronger convergence analysis. Hence the analysis is done twice, once for the general…
Proper continued fractions are generalized continued fractions with positive integer numerators $a_i$ and integer denominators with $b_i\geq a_i$. In this paper we study the strength of approximation of irrational numbers to their…
Gradient schemes is a framework which enables the unified convergence analysis of many different methods -- such as finite elements (conforming, non-conforming and mixed) and finite volumes methods -- for $2^{\rm nd}$ order diffusion…
Using backpropagation to compute gradients of objective functions for optimization has remained a mainstay of machine learning. Backpropagation, or reverse-mode differentiation, is a special case within the general family of automatic…
An appealing property of the natural gradient is that it is invariant to arbitrary differentiable reparameterizations of the model. However, this invariance property requires infinitesimal steps and is lost in practical implementations with…
Threshold methods are popular for ordinal regression problems, which are classification problems for data with a natural ordinal relation. They learn a one-dimensional transformation (1DT) of observations of the explanatory variable, and…
This paper can be seen as an attempt of rethinking the {\em Extra-Gradient Philosophy} for solving Variational Inequality Problems. We show that the properly defined {\em Reduced Gradients} can be used instead for finding approximate…
We propose theoretical analyses of a modified natural gradient descent method in the neural network function space based on the eigendecompositions of neural tangent kernel and Fisher information matrix. We firstly present analytical…
We introduce the concept of inexact first-order oracle of degree q for a possibly nonconvex and nonsmooth function, which naturally appears in the context of approximate gradient, weak level of smoothness and other situations. Our…
This paper introduces a novel inexact gradient descent method with momentum (IGDm) considered as a general framework for various first-order methods with momentum. This includes, in particular, the inexact proximal point method (IPPm),…