Related papers: Multi-Axis and Multi-Vector Gradient Estimations: …

We consider the problem of minimizing the sum of two convex functions: one is the average of a large number of smooth component functions, and the other is a general convex function that admits a simple proximal mapping. We assume the whole…

Situations of a functional predictor paired with a scalar response are increasingly encountered in data analysis. Predictors are often appropriately modeled as square integrable smooth random functions. Imposing minimal assumptions on the…

Reinforcement learning methods for robotics are increasingly successful due to the constant development of better policy gradient techniques. A precise (low variance) and accurate (low bias) gradient estimator is crucial to face…

Many popular feature-attribution methods for interpreting deep neural networks rely on computing the gradients of a model's output with respect to its inputs. While these methods can indicate which input features may be important for the…

Recent studies have shown that fractional calculus is an effective alternative mathematical tool in various scientific fields. However, some investigations indicate that results established in differential and integral calculus do not…

In a variety of problems originating in supervised, unsupervised, and reinforcement learning, the loss function is defined by an expectation over a collection of random variables, which might be part of a probabilistic model or the external…

Are score function estimators an underestimated approach to learning with $k$-subset sampling? Sampling $k$-subsets is a fundamental operation in many machine learning tasks that is not amenable to differentiable parametrization, impeding…

In general, gradient estimates are very important and necessary for deriving convergence results in different geometric flows, and most of them are obtained by analytic methods. In this paper, we will apply a stochastic approach to…

The proximal gradient method is a splitting algorithm for the minimization of the sum of two convex functions, one of which is smooth. It has applications in areas such as mechanics, inverse problems, machine learning, image reconstruction,…

This paper presents enhancement strategies for the Hermitian and skew-Hermitian splitting method based on gradient iterations. The spectral properties are exploited for the parameter estimation, often resulting in a better convergence. In…

This paper proposes several approaches as baselines to compute a shared active subspace for multivariate vector-valued functions. The goal is to minimize the deviation between the function evaluations on the original space and those on the…

We present a new class of gradient-type optimization methods that extends vanilla gradient descent, mirror descent, Riemannian gradient descent, and natural gradient descent. Our approach involves constructing a surrogate for the objective…

Computing the gradient of a function provides fundamental information about its behavior. This information is essential for several applications and algorithms across various fields. One common application that require gradients are…

We study gradient testing and gradient estimation of smooth functions using only a comparison oracle that, given two points, indicates which one has the larger function value. For any smooth $f\colon\mathbb R^n\to\mathbb R$,…

This contribution proposes a new formulation to efficiently compute directional derivatives of order one to fourth. The formulation is based on automatic differentiation implemented with dual numbers. Directional derivatives are particular…

In this paper we study simulation-based methods for estimating gradients in stochastic networks. We derive a new method of calculating weak derivative estimator using importance sampling transform, and our method has less computational cost…

Gradient information on the sampling distribution can be used to reduce the variance of Monte Carlo estimators via Stein's method. An important application is that of estimating an expectation of a test function along the sample path of a…

By providing a simple and efficient way of computing low-variance gradients of continuous random variables, the reparameterization trick has become the technique of choice for training a variety of latent variable models. However, it is not…

We rewrite the standard nodal virtual element method as a generalised gradient method. This re-formulation allows for computing a reliable and efficient error estimator by locally reconstructing broken fluxes and potentials on elemental…

Classically, determining the gradient of a black-box function f:R^p->R requires p+1 evaluations. Using the quantum Fourier transform, two evaluations suffice. This is based on the approximate local periodicity of exp(2*pi*i*f(x)). It is…