Related papers: Algorithmic Differentiation of Linear Algebra Func…
Backpropagation algorithm is the cornerstone for neural network analysis. Paper extends it for training any derivatives of neural network's output with respect to its input. By the dint of it feedforward networks can be used to solve or…
Tenfold improvements in computation speed can be brought to the alternating direction method of multipliers (ADMM) for Semidefinite Programming with virtually no decrease in robustness and provable convergence simply by projecting…
Many problems in physics, chemistry and other fields are perturbative in nature, i.e. differ only slightly from related problems with known solutions. Prominent among these is the eigenvalue perturbation problem, wherein one seeks the…
This survey describes probabilistic algorithms for linear algebra computations, such as factorizing matrices and solving linear systems. It focuses on techniques that have a proven track record for real-world problem instances. The paper…
We present a unified treatment of the Fourier spectra of spherically symmetric nonlocal diffusion operators. We develop numerical and analytical results for the class of kernels with weak algebraic singularity as the distance between source…
A diffusion probabilistic model (DPM) is a generative model renowned for its ability to produce high-quality outputs in tasks such as image and audio generation. However, training DPMs on large, high-dimensional datasets such as…
Alternating direction method of multiplier (ADMM) is a popular method used to design distributed versions of a machine learning algorithm, whereby local computations are performed on local data with the output exchanged among neighbors in…
Many modern computer vision and machine learning applications rely on solving difficult optimization problems that involve non-differentiable objective functions and constraints. The alternating direction method of multipliers (ADMM) is a…
The univariate dimension reduction (UDR) method stands as a way to estimate the statistical moments of the output that is effective in a large class of uncertainty quantification (UQ) problems. UDR's fundamental strategy is to approximate…
The Riemann-Liouville formula for fractional derivatives and integrals (differintegration) is used to derive formulae for matrix order derivatives and integrals. That is, the parameter for integration and differentiation is allowed to…
Motivated by gradient methods in optimization theory, we give methods based on $\psi$-fractional derivatives of order $\alpha$ in order to solve unconstrained optimization problems. The convergence of these methods is analyzed in detail.…
Douglas-Rachford splitting and its equivalent dual formulation ADMM are widely used iterative methods in composite optimization problems arising in control and machine learning applications. The performance of these algorithms depends on…
Alternating Direction Method of Multipliers (ADMM) is a widely used tool for machine learning in distributed settings, where a machine learning model is trained over distributed data sources through an interactive process of local…
Using the notion of order convergent nets, we develop an order-theoretic approach to differentiable functions on Archimedean complex $\Phi$-algebras. Most notably, we improve the Cauchy-Hadamard formulas for universally complete complex…
Solving partial differential equations (PDEs) efficiently is essential for analyzing complex physical systems. Recent advancements in leveraging deep learning for solving PDE have shown significant promise. However, machine learning…
A fast algorithm (linear in the degrees of freedom) for the solution of linear variable-coefficient rational-order fractional integral and differential equations is described. The approach is related to the ultraspherical method for…
Automatic Differentiation (AD) is a powerful tool that allows calculating derivatives of implemented algorithms with respect to all of their parameters up to machine precision, without the need to explicitly add any additional functions.…
Alternating direction method of multiplier (ADMM) is a powerful method to solve decentralized convex optimization problems. In distributed settings, each node performs computation with its local data and the local results are exchanged…
Currently, widely used first-order deep learning optimizers include non-adaptive learning rate optimizers and adaptive learning rate optimizers. The former is represented by SGDM (Stochastic Gradient Descent with Momentum), while the latter…
This article examines the design of Quadratic Fisher Discriminants (QFDs) that operate directly on image pixels, when image ensembles are taken to comprise all rotated and reflected versions of distinct sample images. A procedure based on…