Related papers: Adjoints and Automatic (Algorithmic) Differentiati…
This article aims to demonstrate and discuss the applications of automatic differentiation (AD) for finding derivatives in PDE-constrained optimization problems and Jacobians in non-linear finite element analysis. The main idea is to…
Scientific studies often require the precise calculation of derivatives. In many cases an analytical calculation is not feasible and one resorts to evaluating derivatives numerically. These are error-prone, especially for higher-order…
A large class of non-smooth practical optimization problems can be written as minimization of a sum of smooth and partly smooth functions. We examine such structured problems which also depend on a parameter vector and study the problem of…
The successes of deep learning, variational inference, and many other fields have been aided by specialized implementations of reverse-mode automatic differentiation (AD) to compute gradients of mega-dimensional objectives. The AD…
Approximate computing is a research area where we investigate a wide spectrum of techniques to trade off computation accuracy for better performance or energy consumption. In this work, we provide a general introduction to approximate…
In recent years, the use of adjoint vectors in Computational Fluid Dynamics (CFD) has seen a dramatic rise. Their utility in numerous applications, including design optimization, data assimilation, and mesh adaptation has sparked the…
This article introduces the groundbreaking concept of the financial differential machine learning algorithm through a rigorous mathematical framework. Diverging from existing literature on financial machine learning, the work highlights the…
Automatic differentiation (AD), a technique for constructing new programs which compute the derivative of an original program, has become ubiquitous throughout scientific computing and deep learning due to the improved performance afforded…
Algorithmic differentiation (AD) is a set of techniques that provide partial derivatives of computer-implemented functions. Such a function can be supplied to state-of-the-art AD tools via its source code, or via an intermediate…
Optimizing shapes and topology of physical devices is crucial for both scientific and technological advancements, given its wide-ranging implications across numerous industries and research areas. Innovations in shape and topology…
Inverse design of complex flows is notoriously challenging because of the high cost of high dimensional optimization. Usually, optimization problems are either restricted to few control parameters, or adjoint-based approaches are used to…
Algorithmic Differentiation (AD) can be used to automate the generation of derivatives in arbitrary software projects. This will generate maintainable derivatives, that are always consistent with the computation of the software. If a domain…
First-order optimization algorithms, often preferred for large problems, require the gradient of the differentiable terms in the objective function. These gradients often involve linear operators and their adjoints, which must be applied…
Many engineering problems involve learning hidden dynamics from indirect observations, where the physical processes are described by systems of partial differential equations (PDE). Gradient-based optimization methods are considered…
For scientific machine learning tasks with a lot of custom code, picking the right Automatic Differentiation (AD) system matters. Our Julia package DifferentiationInterface$.$jl provides a common frontend to a dozen AD backends, unlocking…
We present semantic correctness proofs of automatic differentiation (AD). We consider a forward-mode AD method on a higher-order language with algebraic data types and we characterise it as the unique structure-preserving macro given a…
Gradient-based techniques are becoming increasingly critical in quantitative fields, notably in statistics and computer science. The utility of these techniques, however, ultimately depends on how efficiently we can evaluate the derivatives…
Derivatives of differential equation solutions are commonly for parameter estimation, fitting neural differential equations, and as model diagnostics. However, with a litany of choices and a Cartesian product of potential methods, it can be…
The efficient method for computing the sensitivities is the adjoint method. The cost of solving an adjoint equation is comparable to the cost of solving the governing equation. Once the adjoint solution is obtained, the sensitivities to any…
Federated learning (FL) is a general framework for learning across an axis of group partitioned data (heterogeneous clients) while preserving data privacy, under the orchestration of a central server. FL methods often compute gradients of…