Related papers: Striding Across Reynolds Numbers: Representation G…
The practice of deep learning has shown that neural networks generalize remarkably well even with an extreme number of learned parameters. This appears to contradict traditional statistical wisdom, in which a trade-off between model…
Recently, physics-driven deep learning methods have shown particular promise for the prediction of physical fields, especially to reduce the dependency on large amounts of pre-computed training data. In this work, we target the…
We propose the geometry-informed neural operator (GINO), a highly efficient approach to learning the solution operator of large-scale partial differential equations with varying geometries. GINO uses a signed distance function and…
Deep learning methods, which have found successful applications in fields like image classification and natural language processing, have recently been applied to source code analysis too, due to the enormous amount of freely available…
Unsupervised text encoding models have recently fueled substantial progress in NLP. The key idea is to use neural networks to convert words in texts to vector space representations based on word positions in a sentence and their contexts,…
Autoregressive next-step prediction models have become the de-facto standard for building data-driven neural solvers to forecast time-dependent partial differential equations (PDEs). Denoise training that is closely related to diffusion…
Traditional Reynolds-averaged Navier-Stokes (RANS) equations often struggle to predict separated flows accurately. Recent studies have employed data-driven methods to enhance predictions by modifying baseline equations, such as field…
Grokking in transformers trained on algorithmic tasks is characterized by a long delay between training-set fit and abrupt generalization, but the source of that delay remains poorly understood. In encoder-decoder arithmetic models, we…
We investigate geometric regularization strategies for learned latent representations in encoder--decoder reduced-order models. In a fixed experimental setting for the advection--diffusion--reaction (ADR) equation, we model latent dynamics…
Approximate solutions of partial differential equations (PDEs) obtained by neural networks are highly affected by hyper parameter settings. For instance, the model training strongly depends on loss function design, including the choice of…
Navigation foundation models trained on massive webscale data enable agents to generalize across diverse environments and embodiments. However, these models trained solely on offline data, often lack the capacity to reason about the…
We adopt the stretched spiral vortex sub-grid model for large-eddy simulation (LES) of turbulent convection at extreme Rayleigh numbers. We simulate Rayleigh-B\'enard convection (RBC) for Rayleigh numbers ranging from $10^6$ to $10^15$ and…
Large-scale distributed training of deep neural networks suffer from the generalization gap caused by the increase in the effective mini-batch size. Previous approaches try to solve this problem by varying the learning rate and batch size…
Current deep neural networks are highly overparameterized (up to billions of connection weights) and nonlinear. Yet they can fit data almost perfectly through variants of gradient descent algorithms and achieve unexpected levels of…
The adoption of continuous shrinkage priors in high-dimensional linear models has gained widespread attention due to their practical and theoretical advantages. Among them, the R2D2 prior has gained popularity for its intuitive…
This paper presents a method for modeling transient fluid flow in subsurface reservoir systems based on the developed neural operator architecture (TFNO-opt). Reservoir systems are complex dynamic objects with distributed parameters…
The background of this work is the problem of reducing the aerodynamic turbulent friction drag, which is an important source of energy waste in innumerable technological fields. We develop a theoretical framework aimed at predicting the…
Latest diffusion models have shown promising results in category-level 6D object pose estimation by modeling the conditional pose distribution with depth image input. The existing methods, however, suffer from slow convergence during…
Generalizability of machine-learning (ML) based turbulence closures to accurately predict unseen practical flows remains an important challenge. At the Reynolds-averaged Navier-Stokes (RANS) level, NN-based turbulence closure modeling is…
Reliable multiscale models of thrombosis require platelet-scale fidelity at organ-scale cost, a gap that scientific machine learning has the potential to narrow. We train a DeepONet surrogate on platelet dynamics generated with LAMMPS for…