Related papers: Consistent Computation of First- and Second-Order …
We deal with the problem of gradient estimation for stochastic differentiable relaxations of algorithms, operators, simulators, and other non-differentiable functions. Stochastic smoothing conventionally perturbs the input of a…
Specialized function gradient computing hardware could greatly improve the performance of state-of-the-art optimization algorithms, e.g., based on gradient descent or conjugate gradient methods that are at the core of control, machine…
Finite-sum optimization problems are ubiquitous in machine learning, and are commonly solved using first-order methods which rely on gradient computations. Recently, there has been growing interest in \emph{second-order} methods, which rely…
We propose new formulations of geometric curvature flows -- referred to as \emph{dual formulations} -- that are equivalent to the original formulations but provide a novel framework for constructing linearly implicit and energy-stable…
We generate random functions locally via a novel generalization of Dyson Brownian motion, such that the functions are in a desired differentiability class, while ensuring that the Hessian is a member of the Gaussian orthogonal ensemble…
Stochastic gradients for deep neural networks exhibit strong correlations along the optimization trajectory, and are often aligned with a small set of Hessian eigenvectors associated with outlier eigenvalues. Recent work shows that…
Almost universally in computer vision, when surface derivatives are required, they are computed using only first order accurate finite difference approximations. We propose a new method for computing numerical derivatives based on 2D…
The Gaussian curvature of a two-dimensional Riemannian manifold is uniquely determined by the choice of the metric. The formulas for computing the curvature in terms of components of the metric, in isothermal coordinates, involve the…
Generative models that produce point clouds have emerged as a powerful tool to represent 3D surfaces, and the best current ones rely on learning an ensemble of parametric representations. Unfortunately, they offer no control over the…
To target challenges in differentiable optimization we analyze and propose strategies for derivatives of the Mat\'ern kernel with respect to the smoothness parameter. This problem is of high interest in Gaussian processes modelling due to…
Many developments in Mathematics involve the computation of higher order derivatives of Gaussian density functions. The analysis of univariate Gaussian random variables is a well-established field whereas the analysis of their multivariate…
The accurate computation of the covariance matrix of fitted model parameters is a somewhat neglected task in Statistics. Algorithms are given for computing accurate covariance matrices derived from computing the Hessian matrix by numerical…
Discrete differential geometry aims to develop discrete equivalents of the geometric notions and methods of classical differential geometry. In this survey we discuss the following two fundamental Discretization Principles: the…
We present a systematic derivation of the algorithms required for computing the gradient and the action of the Hessian of an arbitrary misfit function for large-scale parameter estimation problems involving linear time-dependent PDEs with…
Variational representations of divergences and distances between high-dimensional probability distributions offer significant theoretical insights and practical advantages in numerous research areas. Recently, they have gained popularity in…
Motivated by important applications in image processing, we study a class of second-order geometric quasilinear hyperbolic partial differential equations (PDEs). This is inspired by the recent development of second-order damping systems…
This paper establishes the basis of the quaternionic differential geometry ($\mathbbm H$DG) initiated in a previous article. The usual concepts of curves and surfaces are generalized to quaternionic constraints, as well as the curvature and…
Finite differences have been widely used in mathematical theory as well as in scientific and engineering computations. These concepts are constantly mentioned in calculus. Most frequently-used difference formulas provide excellent…
A heuristic formula for 5-point approximation of the first derivative of an unknown function whose values are measured with an error at unequally spaced points is proposed. The derivative at a given point is calculated using the effective…
This paper is devoted to studying the first-order variational analysis of non-convex and non-differentiable functions that may not be subdifferentially regular. To achieve this goal, we entirely rely on two concepts of directional…