相关论文: Backpropagation in hyperbolic chaos via adjoint sh…

We devise the fast adjoint response algorithm for the gradient of physical measures (long-time-average statistics) of discrete-time hyperbolic chaos with respect to many system parameters. Its cost is independent of the number of…

For hyperbolic diffeomorphisms, we define adjoint shadowing directions as a bounded inhomogeneous adjoint solution whose initial condition has zero component in the unstable adjoint direction. For hyperbolic flows, we define adjoint…

Nonintrusive shadowing algorithms efficiently compute $v$, the difference between shadowing trajectories, then use $v$ to compute derivatives of averaged objectives of chaos with respect to parameters of the dynamical system. However,…

Adjoint-based sensitivity analysis is of interest in computational science due to its ability to compute sensitivities at a lower cost with respect to several design parameters. However, conventional sensitivity analysis methods fail in the…

In a uniformly hyperbolic system, we consider the problem of finding the optimal infinitesimal perturbation to apply to the system, from a certain set $P$ of feasible ones, to maximally increase the expectation of a given observation…

The sensitivity of long-time averages of a hyperbolic chaotic system to parameter perturbations can be determined using the shadowing direction, the uniformly-bounded-in-time solution of the sensitivity equations. Although its existence is…

Chaotic dynamical systems are characterized by the sensitive dependence of trajectories on initial conditions. Conventional sensitivity analysis of time-averaged functionals yields unbounded sensitivities when the simulation is chaotic. The…

Models with dominant advection always posed a difficult challenge for projection-based reduced order modelling. Many methodologies that have recently been proposed are based on the pre-processing of the full-order solutions to accelerate…

Backpropagation algorithm is the cornerstone for neural network analysis. Paper extends it for training any derivatives of neural network's output with respect to its input. By the dint of it feedforward networks can be used to solve or…

In this paper, we investigate the limiting behavior of a continuous-time counterpart of the Stochastic Gradient Descent (SGD) algorithm applied to two-layer overparameterized neural networks, as the number or neurons (ie, the size of the…

If a nonnegative selfadjoint linear relation $A$ in a Hilbert space and a closed subspace $\mathcal{S}$ are assumed to satisfy that the domain of $A$ is invariant under the orthogonal projector onto $\mathcal{S},$ then $A$ admits a…

This paper presents a concise mathematical framework for investigating both feed-forward and backward process, during the training to learn model weights, of an artificial neural network (ANN). Inspired from the idea of the two-step rule…

We derive and prove the `fast response' formula for the linear response, the parameter derivatives of long-time-averaged statistics, of hyperbolic deterministic chaotic systems. The expression is pointwisely defined so we can compute the…

Embedding discrete solvers as differentiable layers has given modern deep learning architectures combinatorial expressivity and discrete reasoning capabilities. The derivative of these solvers is zero or undefined, therefore a meaningful…

We study the asymptotic behavior of Lipschitz continuous solutions of nonlinear degenerate parabolic equations in the periodic setting. Our results apply to a large class of Hamilton-Jacobi-Bellman equations. Defining S as the set where the…

We show that linear chaos in the space $c(\mathbb{N})$ of convergent sequences cannot be arrived at by merely extending the weighted backward shifts in the space $c_0(\mathbb{N})$ of vanishing sequences. Applying a newly found sufficient…

We analyze the convergence of the one-level overlapping domain decomposition preconditioner SORAS (Symmetrized Optimized Restricted Additive Schwarz) applied to a generic linear system whose matrix is not necessarily symmetric/self-adjoint…

We introduce backdrop, a flexible and simple-to-implement method, intuitively described as dropout acting only along the backpropagation pipeline. Backdrop is implemented via one or more masking layers which are inserted at specific points…

A recent paper by Boughammoura (2023) describes the back-propagation algorithm in terms of an alternative formulation called the F-adjoint method. In particular, by the F-adjoint algorithm the computation of the loss gradient, with respect…

This paper presents a groundbreaking advancement in the theory of operators defined on octonionic Hilbert spaces, successfully resolving a fundamental challenge that has persisted for over six decades. Due to the intrinsic non-associative…