Related papers: An adjoint method for training data-driven reduced…

State-of-the-art training algorithms for deep learning models are based on stochastic gradient descent (SGD). Recently, many variations have been explored: perturbing parameters for better accuracy (such as in Extragradient), limiting SGD…

Scalable machine learning over big data is an important problem that is receiving a lot of attention in recent years. On popular distributed environments such as Hadoop running on a cluster of commodity machines, communication costs are…

We propose Adaptive Compressed Gradient Descent (AdaCGD) - a novel optimization algorithm for communication-efficient training of supervised machine learning models with adaptive compression level. Our approach is inspired by the recently…

Theory and methods to obtain parametric reduced-order models by moment matching are presented. The definition of the parametric moment is introduced, and methods (model-based and data-driven) for the approximation of the parametric moment…

An algorithm is proposed for solving optimization problems arising in neural network training for supervised learning. The unique feature of the algorithm is the use of an auxiliary loss, in addition to the original loss employed for model…

A recent paper by Boughammoura (2023) describes the back-propagation algorithm in terms of an alternative formulation called the F-adjoint method. In particular, by the F-adjoint algorithm the computation of the loss gradient, with respect…

Join ordering is the NP-hard problem of selecting the most efficient order in which to evaluate joins (conjunctive, binary operators) in a database query. Because query execution performance critically depends on this choice, join ordering…

The importance of state estimation in fluid mechanics is well-established; it is required for accomplishing several tasks including design/optimization, active control, and future state prediction. A common tactic in this regards is to rely…

Predictive modeling involving simulation and sensor data at the same time, is a growing challenge in computational science. Even with large-scale finite element models, a mismatch to the sensor data often remains, which can be attributed to…

A new method for data-driven interpolatory model reduction is presented in this paper. Using the so-called data informativity perspective, we define a framework that enables the computation of moments at given (possibly complex)…

We establish that nonconvex definable parametric optimization problems with possibly nonsmooth objectives, inequality constraints, conic constraint systems, and non-unique primal and dual solutions admit an adjoint state formula under a…

Stochastic Gradient Descent (SGD) methods see many uses in optimization problems. Modifications to the algorithm, such as momentum-based SGD methods have been known to produce better results in certain cases. Much of this, however, is due…

Asynchronous distributed algorithms are a popular way to reduce synchronization costs in large-scale optimization, and in particular for neural network training. However, for nonsmooth and nonconvex objectives, few convergence guarantees…

A high-order accurate adjoint-based optimization framework is presented for unsteady multiphysics problems. The fully discrete adjoint solver relies on the high-order, linearly stable, partitioned solver introduced in [1], where different…

Imaging Earth structure or seismic sources from seismic data involves minimizing a target misfit function, and is commonly solved through gradient-based optimization. The adjoint-state method has been developed to compute the gradient…

There is a critical need for efficient and reliable active flow control strategies to reduce drag and noise in aerospace and marine engineering applications. While traditional full-order models based on the Navier-Stokes equations are not…

Distributionally robust optimization (DRO) problems are increasingly seen as a viable method to train machine learning models for improved model generalization. These min-max formulations, however, are more difficult to solve. We therefore…

Gradient descent is one of the most widely used iterative algorithms in modern statistical learning. However, its precise algorithmic dynamics in high-dimensional settings remain only partially understood, which has limited its broader…

This paper offers a data-driven approach for designing adaptive suboptimal second-order sliding mode (ASSOSM) controllers for single-input nonlinear systems, characterized by perturbed strict-feedback structures with unknown dynamics. The…

Data-driven model reduction methods provide a nonintrusive way of constructing computationally efficient surrogates of high-fidelity models for real-time control of soft robots. This work leverages the Lagrangian nature of the model…