Related papers: The GNAT method for nonlinear model reduction: eff…

Machine-learning-based surrogate models offer significant computational efficiency and faster simulations compared to traditional numerical methods, especially for problems requiring repeated evaluations of partial differential equations.…

Minimizing loss functions is central to machine-learning training. Although first-order methods dominate practical applications, higher-order techniques such as Newton's method can deliver greater accuracy and faster convergence, yet are…

Accounting for inequality constraints, such as boundedness, monotonicity or convexity, is challenging when modeling costly-to-evaluate black box functions. In this regard, finite-dimensional Gaussian process (GP) regression models bring a…

One of the very few mathematically rigorous nonlinear model reduction methods is the restriction of a dynamical system to a low-dimensional, sufficiently smooth, attracting invariant manifold. Such manifolds are usually found using local…

Following early work on Hessian-free methods for deep learning, we study a stochastic generalized Gauss-Newton method (SGN) for training DNNs. SGN is a second-order optimization method, with efficient iterations, that we demonstrate to…

The rise of neural network (NN) applications has prompted an increased interest in compression, with a particular focus on channel pruning, which does not require any additional hardware. Most pruning methods employ either single-layer…

Physics-guided deep learning is an important prevalent research topic in scientific machine learning, which has tremendous potential in various complex applications including science and engineering. In these applications, data is expensive…

In this paper, we present CT-AGD (Curvature-Tuned Accelerated Gradient Descent), an optimization method for non-convex optimization problems in deep learning training tasks. CT-AGD is a general boosting procedure that accelerates…

Recent trends in lower precision, e.g. half-precision floating point, training have shown improved system performance and reduced memory usage for Deep Learning while maintaining accuracy. However, current GNN systems cannot achieve such…

In this manuscript, we propose a general proximal quasi-Newton method tailored for nonconvex and nonsmooth optimization problems, where we do not require the sequence of the variable metric (or Hessian approximation) to be uniformly bounded…

Computational Fluid Dynamics (CFD) is a hugely important subject with applications in almost every engineering field, however, fluid simulations are extremely computationally and memory demanding. Towards this end, we present Lat-Net, a…

The Projected Gradient Descent (PGD) algorithm is a widely used and efficient first-order method for solving constrained optimization problems due to its simplicity and scalability in large design spaces. Building on recent advancements in…

Uncovering rationales behind predictions of graph neural networks (GNNs) has received increasing attention over the years. Existing literature mainly focus on selecting a subgraph, through combinatorial optimization, to provide faithful…

Achievement of solutions in Navier-Stokes equation is one of challenging quests, especially for its closure problem. For achievement of particular solutions, there are variety of numerical simulations including Direct Numerical Simulation…

The G-equation is a well-known model for studying front propagation in turbulent combustion. In this paper, we develop an efficient model reduction method for computing \textcolor{black}{regular solutions} of viscous G-equations in…

The rapid development of deep learning has significant implications for the advancement of Computational Fluid Dynamics (CFD). Currently, most pixel-grid-based deep learning methods for flow field prediction exhibit significantly reduced…

This paper presents the design and evaluation of a GPU-accelerated inference pipeline for transformer models using NVIDIA TensorRT with mixed-precision optimization. We evaluate BERT-base (110M parameters) and GPT-2 (124M parameters) across…

A significant advancement in nonlinear projection-based model order reduction (PMOR) is presented through a highly effective methodology. This methodology employs Gaussian process regression (GPR) and radial basis function (RBF)…

A nonlocal subgrid-scale stress (SGS) model is developed based on the convolution neural network (CNN), a powerful supervised data-driven approach. The CNN is an ideal approach to naturally consider nonlocal spatial information in…

Graph Neural Networks (GNNs) are emerging as powerful tools for nonlinear Model Order Reduction (MOR) of time-dependent parameterized Partial Differential Equations (PDEs). However, existing methodologies struggle to combine geometric…