Related papers: Conservative set valued fields, automatic differen…
This paper presents some applications using recently developed algorithms for smooth-continuous data reconstruction based on the digital-discrete method. The classical discrete method for data reconstruction is based on domain decomposition…
The ``differentiability gap'' presents a primary bottleneck in Earth system deep learning: since models cannot be trained directly on non-differentiable scientific metrics and must rely on smooth proxies (e.g., MSE), they often fail to…
While backpropagation--reverse-mode automatic differentiation--has been extraordinarily successful in deep learning, it requires two passes (forward and backward) through the neural network and the storage of intermediate activations.…
While normalizing flows for continuous data have been extensively researched, flows for discrete data have only recently been explored. These prior models, however, suffer from limitations that are distinct from those of continuous flows.…
For a real function, automatic differentiation is such a standard algorithm used to efficiently compute its gradient, that it is integrated in various neural network frameworks. However, despite the recent advances in using complex…
Many engineering problems involve learning hidden dynamics from indirect observations, where the physical processes are described by systems of partial differential equations (PDE). Gradient-based optimization methods are considered…
Enforcing exact macroscopic conservation laws, such as mass and energy, in neural partial differential equation (PDE) solvers is computationally challenging in high dimensions. Traditional discrete projections rely on deterministic…
The trace formula for the evolution operator associated with nonlinear stochastic flows with weak additive noise is cast in the path integral formalism. We integrate over the neighborhood of a given saddlepoint exactly by means of a smooth…
In this paper, we establish a large deviation principle for the conservative stochastic partial differential equations, whose solutions are related to stochastic differential equations with interaction. The weak convergence method and the…
Inspired by Gauss-Newton-like methods, we study the benefit of leveraging the structure of deep learning objectives, namely, the composition of a convex loss function and of a nonlinear network, in order to derive better direction oracles…
Many conservative partial differential equations such as the Korteweg-de Vries (KdV) equation, and the nonlinear Schr\"{o}dinger equations, the Klein-Gordon equation have more than one invariant functionals. In this paper, we propose the…
We prove closed-form equations for the exact high-dimensional asymptotics of a family of first order gradient-based methods, learning an estimator (e.g. M-estimator, shallow neural network, ...) from observations on Gaussian data with…
Many important physical systems can be described as the evolution of a Hamiltonian system, which has the important property of being conservative, that is, energy is conserved throughout the evolution. Physics Informed Neural Networks and…
We study a class of semi-implicit Taylor-type numerical methods that are easy to implement and designed to solve multidimensional stochastic differential equations driven by a general rough noise, e.g. a fractional Brownian motion. In the…
The aim of this work is to apply a semi-implicit (SI) strategy within a Rosenbrock-type and IMEX linear multistep (LM) framework to a sequence of 1D time-dependent partial differential equations (PDEs) with high order spatial derivatives.…
We introduce a novel generative formulation of deep probabilistic models implementing "soft" constraints on their function dynamics. In particular, we develop a flexible methodological framework where the modeled functions and derivatives…
We introduce a real vector space composed of set-valued maps on an open set X and note it by S. It is a complete metric space and a complete lattice. The set of continuous functions on X is dense in S as in a metric space and as in a…
A certain representation for the Heisenberg algebra in finite-difference operators is established. The Lie-algebraic procedure of discretization of differential equations with isospectral property is proposed. Using $sl_2$-algebra based…
We develop a compositional approach for automatic and symbolic differentiation based on categorical constructions in functional analysis where derivatives are linear functions on abstract vectors rather than being limited to scalars,…
The asymptotic behavior of the stochastic gradient algorithm with a biased gradient estimator is analyzed. Relying on arguments based on the dynamic system theory (chain-recurrence) and the differential geometry (Yomdin theorem and…