Related papers: Controlling Statistical, Discretization, and Trunc…
Operator learning aims to discover properties of an underlying dynamical system or partial differential equation (PDE) from data. Here, we present a step-by-step guide to operator learning. We explain the types of problems and PDEs amenable…
In \cite{liu2022practical}, a general algorithm is developed to efficiently obtain the best accuracy using the regular refinement. The adaptive refinement allows for obtaining an accuracy with a smaller number of DoFs compared with the…
Implementing linear transformations is a key task in the decentralized signal processing framework, which performs learning tasks on data sets distributed over multi-node networks. That kind of network can be represented by a graph.…
Fourier Neural Operators (FNOs) have emerged as promising surrogates for partial differential equation solvers. In this work, we extensively tested FNOs on a variety of systems with non-linear and non-stationary properties, using a wide…
We study how different output layer parameterizations of a deep neural network affects learning and forgetting in continual learning settings. The following three effects can cause catastrophic forgetting in the output layer: (1) weights…
We propose theoretical analyses of a modified natural gradient descent method in the neural network function space based on the eigendecompositions of neural tangent kernel and Fisher information matrix. We firstly present analytical…
We study in this paper lower bounds for the generalization error of models derived from multi-layer neural networks, in the regime where the size of the layers is commensurate with the number of samples in the training data. We show that…
Fourier neural operators (FNOs) provide a mesh-independent way to learn solution operators for partial differential equations, yet their efficacy for magnetized turbulence is largely unexplored. Here we train an FNO surrogate for the 2-D…
Follow-the-Regularized-Leader (FTRL) algorithms are a popular class of learning algorithms for online linear optimization (OLO) that guarantee sub-linear regret, but the choice of regularizer can significantly impact dimension-dependent…
In this article, we develop comprehensive frequency domain methods for estimating and inferring the second-order structure of spatial point processes. The main element here is on utilizing the discrete Fourier transform (DFT) of the point…
We study the performance of machine learning binary classification techniques in terms of error probabilities. The statistical test is based on the Data-Driven Decision Function (D3F), learned in the training phase, i.e., what is…
Learning operators between infinitely dimensional spaces is an important learning task arising in wide applications in machine learning, imaging science, mathematical modeling and simulations, etc. This paper studies the nonparametric…
We study the generalization error of statistical learning models in a Federated Learning (FL) setting. Specifically, there are $K$ devices or clients, each holding an independent own dataset of size $n$. Individual models, learned locally…
The first result of applying the machine/deep learning technique to the fluid closure problem is presented in this paper. As a start, three different types of neural networks (multilayer perceptron (MLP), convolutional neural network (CNN)…
Empirical studies show that gradient-based methods can learn deep neural networks (DNNs) with very good generalization performance in the over-parameterization regime, where DNNs can easily fit a random labeling of the training data. Very…
We formulate mold filling in metal casting as a 2D neural operator learning problem that maps geometry and boundary data on an unstructured mesh to time resolved flow quantities, replacing expensive transient CFD. In the proposed method, a…
When optimizing over-parameterized models, such as deep neural networks, a large set of parameters can achieve zero training error. In such cases, the choice of the optimization algorithm and its respective hyper-parameters introduces…
A significant advance in accelerating neural network training has been the development of normalization methods, permitting the training of deep models both faster and with better accuracy. These advances come with practical challenges: for…
We reconstruct the velocity field of incompressible flows given a finite set of measurements. For the spatial approximation, we introduce the Sparse Fourier divergence-free (SFdf) approximation based on a discrete $L^2$ projection. Within…
Fourier Neural Operators are deep learning models that learn mappings between function spaces and can be used to learn and solve partial differential equations (PDEs), in some cases significantly faster than traditional PDE solvers. Within…