Related papers: Multi-Axis and Multi-Vector Gradient Estimations: …
The gradient scheme framework is based on a small number of properties and encompasses a large number of numerical methods for diffusion models. We recall these properties and develop some new generic tools associated with the gradient…
We present a new method of calculating scalar propagator and vertex functions in the two-loop approximation, for arbitrary masses of particles. It is based on a double integral representation, suitable for numerical evaluation. Real and…
In this paper we consider the approximation of a function by its interpolating multilinear spline and the approximation of its derivatives by the derivatives of the corresponding spline. We derive formulas for the uniform approximation…
This paper is devoted to first-order algorithms for smooth convex optimization with inexact gradients. Unlike the majority of the literature on this topic, we consider the setting of relative rather than absolute inexactness. More…
In this paper, in a multivariate setting we derive near optimal rates of convergence in the minimax sense for estimating partial derivatives of the mean function for functional data observed under a fixed synchronous design over H\"older…
We present a previously unexplored forward-mode differentiation method for Maxwell's equations, with applications in the field of sensitivity analysis. This approach yields exact gradients and is similar to the popular adjoint variable…
Aims. New or refined methods for determining instantaneous scalar and vector pattern speeds from a restricted domain are developed, for applications in N-body simulations or in galaxies. Methods. The general feature used throughout follows…
Stochastic gradient descent (SGD) is the workhorse of modern machine learning. Sometimes, there are many different potential gradient estimators that can be used. When so, choosing the one with the best tradeoff between cost and variance is…
This paper proposes an efficient probabilistic method that computes combinatorial gradient fields for two dimensional image data. In contrast to existing algorithms, this approach yields a geometric Morse-Smale complex that converges almost…
In the recent years, various gradient descent algorithms including the methods of gradient descent, gradient descent with momentum, adaptive gradient (AdaGrad), root-mean-square propagation (RMSProp) and adaptive moment estimation (Adam)…
For real symmetric matrices that are accessible only through matrix vector products, we present Monte Carlo estimators for computing the diagonal elements. Our probabilistic bounds for normwise absolute and relative errors apply to Monte…
We study gradient-based regularization methods for neural networks. We mainly focus on two regularization methods: the total variation and the Tikhonov regularization. Applying these methods is equivalent to using neural networks to solve…
Federated learning heavily relies on distributed gradient descent techniques. In the situation where gradient information is not available, the gradients need to be estimated from zeroth-order information, which typically involves computing…
The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective. However, this technique does not easily apply to commonly used distributions such as beta or gamma…
We consider the approximation of manifold-valued functions by embedding the manifold into a higher dimensional space, applying a vector-valued approximation operator and projecting the resulting vector back to the manifold. It is well known…
We derive and prove an explicit formula for the sum of the fractional parts of certain geometric series. Although the proof is straightforward, we have been unable to locate any reference to this result. This summation formula allows us to…
The problem of approximating a sampled function using sums of a fixed number of complex exponentials is considered. We use alternating projections between fixed rank matrices and Hankel matrices to obtain such an approximation. Convergence,…
The level set approach represents surfaces implicitly, and advects them by evolving a level set function, which is numerically defined on an Eulerian grid. Here we present an approach that augments the level set function values by gradient…
We review and extend a technique for recovering a smooth function from its averages over a wide class of curves in a general region of Euclidean space. The method is based on complexification of the underlying vector fields defining the…
Inverse design of large-area metasurfaces can potentially exploit the full parameter space that such devices offer and achieve highly efficient multifunctional flat optical elements. However, since practically useful flat optics elements…