相关论文: Improved Greedy Identification of Latent Dynamics …

Stochastic gradient descent (SGD) is perhaps the most prevalent optimization method in modern machine learning. Contrary to the empirical practice of sampling from the datasets without replacement and with (possible) reshuffling at each…

Class Incremental Learning (CIL) aims to enable models to learn new classes sequentially while retaining knowledge of previous ones. Although current methods have alleviated catastrophic forgetting (CF), recent studies highlight that the…

In federated learning (FL), model training performance is strongly impacted by data heterogeneity across clients. Client-drift compensation methods have recently emerged as a solution to this issue, introducing correction terms into local…

How can we understand gradient-based training over non-convex landscapes? The edge of stability phenomenon, introduced in Cohen et al. (2021), indicates that the answer is not so simple: namely, gradient descent (GD) with large step sizes…

Gradient Descent (GD) and Conjugate Gradient (CG) methods are among the most effective iterative algorithms for solving unconstrained optimization problems, particularly in machine learning and statistical modeling, where they are employed…

Algebraic or geometric multigrid methods are commonly used in numerical solvers as they are a multi-resolution method able to handle problems with multiple scales. In this work, we propose a modification to the commonly-used U-Net neural…

We study the problem of finding the best linear model that can minimize least-squares loss given a data-set. While this problem is trivial in the low dimensional regime, it becomes more interesting in high dimensions where the population…

The performance of gradient-based optimization methods, such as standard gradient descent (GD), greatly depends on the choice of learning rate. However, it can require a non-trivial amount of user tuning effort to select an appropriate…

In this work, we present a class of new efficient models for water flow in shallow unconfined aquifers, giving an alternative to the classical but less tractable 3D-Richards model. Its derivation is guided by two ambitions: any new model…

It is believed that Gradient Descent (GD) induces an implicit bias towards good generalization in training machine learning models. This paper provides a fine-grained analysis of the dynamics of GD for the matrix sensing problem, whose goal…

Gradient clipping is commonly used in training deep neural networks partly due to its practicability in relieving the exploding gradient problem. Recently, \citet{zhang2019gradient} show that clipped (stochastic) Gradient Descent (GD)…

In this work, we make two improvements on the staggered grid hydrodynamics (SGH) Lagrangian scheme for modeling 2-dimensional compressible multi-material flows on triangular mesh. The first improvement is the construction of a dynamic local…

Bilevel optimization is a fundamental tool in hierarchical decision-making and has been widely applied to machine learning tasks such as hyperparameter tuning, meta-learning, and continual learning. While significant progress has been made…

MHD turbulence is likely to play an important role in several astrophysical scenarios where the magnetic Reynolds is very large. Numerically, these cases can be studied efficiently by means of Large Eddy Simulations, in which the…

Stochastic Gradient Descent (SGD) is an out-of-equilibrium algorithm used extensively to train artificial neural networks. However very little is known on to what extent SGD is crucial for to the success of this technology and, in…

In this project, we propose a Variational Inference algorithm to approximate posterior distributions. Building on prior methods, we develop the Gradient-Steered Stein Variational Gradient Descent (G-SVGD) approach. This method introduces a…

Traditional fluid flow predictions require large computational resources. Despite recent progress in parallel and GPU computing, the ability to run fluid flow predictions in real-time is often infeasible. Recently developed machine learning…

While methods exist for aligning flow matching models--a popular and effective class of generative models--with human preferences, existing approaches fail to achieve both adaptation efficiency and probabilistically sound prior…

The remarkable success of pretrain-then-finetune paradigm has led to a proliferation of available pre-trained models for vision tasks. This surge presents a significant challenge in efficiently choosing the most suitable pre-trained models…

We employ the principle of minimum pressure gradient to transform problems in unsteady computational fluid dynamics (CFD) into a convex optimization framework subject to linear constraints. This formulation permits solving, for the first…