Related papers: The Discrete Adjoint Method: Efficient Derivatives…
Perturbation and operator adjoint method are used to give the right adjoint form rigourously. From the derivation, we can have following results: 1) The loss gradient is not an ODE, it is an integral and we shows the reason; 2) The…
To increase the predictive power of a model, one needs to estimate its unknown parameters. Almost all parameter estimation techniques in ordinary differential equation models suffer from either a small convergence region or enormous…
Numerous models for supervised and reinforcement learning benefit from combinations of discrete and continuous model components. End-to-end learnable discrete-continuous models are compositional, tend to generalize better, and are more…
A stochastic conjugate gradient method for approximation of a function is proposed. The proposed method avoids computing and storing the covariance matrix in the normal equations for the least squares solution. In addition, the method…
We propose new numerical approach to non-conservative dynamical systems. Our method being of low order, enhances qualitative performance of standard discrete gradient algorithm, thank to new concept of a reservoir. Paper is of explanatory…
Gradient-based optimization methods are commonly used to identify local optima in high-dimensional spaces. When derivatives cannot be evaluated directly, stochastic estimators can provide approximate gradients. However, these estimators'…
In this paper we introduce discrete gradient methods to discretize irreversible port-Hamiltonian systems showing that the main qualitative properties of the continuous system are preserved using this kind discretizations methods.
In this article we consider an optimization problem where the objective function is evaluated at the fixed-point of a contraction mapping parameterized by a control variable, and optimization takes place over this control variable. Since…
Diffusive representations of fractional derivatives have proven to be useful tools in the construction of fast and memory efficient numerical methods for solving fractional differential equations. A common challenge in many of the known…
In this work, we present an adjoint-based method for discovering the underlying governing partial differential equations (PDEs) given data. The idea is to consider a parameterized PDE in a general form and formulate a PDE-constrained…
Adjoint methods enable the accurate calculation of the sensitivities of a quantity of interest. The sensitivity is obtained by solving the adjoint system, which can be derived by continuous or discrete adjoint strategies. In acoustic wave…
Computation methods for solving entropy-regularized reward optimization -- a class of problems widely used for fine-tuning generative models -- have advanced rapidly. Among those, Adjoint Matching (AM, Domingo-Enrich et al., 2025) has…
We give an introduction to discrete functional analysis techniques for stationary and transient diffusion equations. We show how these techniques are used to establish the convergence of various numerical schemes without assuming…
The discrete gradient methods are integrators designed to preserve invariants of ordinary differential equations. From a formal series expansion of a subclass of these methods, we derive conditions for arbitrarily high order. We derive…
The paper contributes to strengthening the relation between machine learning and the theory of differential equations. In this context, the inverse problem of fitting the parameters, and the initial condition of a differential equation to…
In this paper, we consider gradient methods for minimizing smooth convex functions, which employ the information obtained at the previous iterations in order to accelerate the convergence towards the optimal solution. This information is…
Interpreting gradient methods as fixed-point iterations, we provide a detailed analysis of those methods for minimizing convex objective functions. Due to their conceptual and algorithmic simplicity, gradient methods are widely used in…
We consider time series data modeled by ordinary differential equations (ODEs), widespread models in physics, chemistry, biology and science in general. The sensitivity analysis of such dynamical systems usually requires calculation of…
Stellarators are a promising route to steady-state fusion power. However, to achieve the required confinement, the magnetic geometry must be highly optimized. This optimization requires navigating high-dimensional spaces, often…
Finite differences have been widely used in mathematical theory as well as in scientific and engineering computations. These concepts are constantly mentioned in calculus. Most frequently-used difference formulas provide excellent…