Related papers: Towards a new PGD strategy for the simulation of s…
In this paper, we introduce the tamed stochastic gradient descent method (TSGD) for optimization problems. Inspired by the tamed Euler scheme, which is a commonly used method within the context of stochastic differential equations, TSGD is…
In this paper, we investigate the impact of stochasticity and large stepsizes on the implicit regularisation of gradient descent (GD) and stochastic gradient descent (SGD) over diagonal linear networks. We prove the convergence of GD and…
Stochastic approximation (SA) and stochastic gradient descent (SGD) algorithms are work-horses for modern machine learning algorithms. Their constant stepsize variants are preferred in practice due to fast convergence behavior. However,…
This paper introduces an extended tensor decomposition (XTD) method for model reduction. The proposed method is based on a sparse non-separated enrichment to the conventional tensor decomposition, which is expected to improve the…
Stochastic descent methods (of the gradient and mirror varieties) have become increasingly popular in optimization. In fact, it is now widely recognized that the success of deep learning is not only due to the special deep architecture of…
We derive a large-strain plate model that allows to describe transient, coupled processes involving elasticity and solvent migration, by performing a dimensional reduction of a three-dimensional poroelastic theory. We apply the model to…
The presence of uncertainty in material properties and geometry of a structure is ubiquitous. The design of robust engineering structures, therefore, needs to incorporate uncertainty in the optimization process. Stochastic gradient descent…
The dynamic mode decomposition (DMD) has become a leading tool for data-driven modeling of dynamical systems, providing a regression framework for fitting linear dynamical models to time-series measurement data. We present a simple…
The dynamic mode decomposition (DMD) is a data-driven approach that extracts the dominant features from spatiotemporal data. In this work, we introduce sparse-mode DMD, a new variant of the optimized DMD framework that specifically…
We address the problem of solving strongly convex and smooth minimization problems using stochastic gradient descent (SGD) algorithm with a constant step size. Previous works suggested to combine the Polyak-Ruppert averaging procedure with…
Data-driven dimensionality reduction methods such as proper orthogonal decomposition (POD) and dynamic mode decomposition (DMD) have proven to be useful for exploring complex phenomena within fluid dynamics and beyond. A well-known…
In this paper, we propose a novel one-dimensional (1D) discrete differential geometry (DDG)-based numerical method for geometrically nonlinear mechanics analysis (e.g., buckling and snapping) of axisymmetric shell structures. Our numerical…
Stochastic gradient descent (SGD) algorithm and its variations have been effectively used to optimize neural network models. However, with the rapid growth of big data and deep learning, SGD is no longer the most suitable choice due to its…
In this work, we present a multiscale approach for the reliable coarse-scale approximation of spatial network models represented by a linear system of equations with respect to the nodes of a graph. The method is based on the ideas of the…
This work presents a robust design optimization approach for a char combustion process in a limited-data setting, where simulations of the fluid-solid coupled system are computationally expensive. We integrate a polynomial dimensional…
SketchySGD improves upon existing stochastic gradient methods in machine learning by using randomized low-rank approximations to the subsampled Hessian and by introducing an automated stepsize that works well across a wide range of convex…
Bilevel optimization has been widely used in many machine learning applications such as hyperparameter optimization and meta learning. Recently, many simple stochastic gradient descent(SGD) type algorithms(without using momentum and…
In this paper, we introduce proximal gradient temporal difference learning, which provides a principled way of designing and analyzing true stochastic gradient temporal difference learning algorithms. We show how gradient TD (GTD)…
We apply the Proper Orthogonal Decomposition (POD) method for the efficient simulation of several scenarios undergone by Micro-Electro-Mechanical-Systems, involving nonlinearites of geometric and electrostatic nature. The former type of…
Stochastic dynamical systems with continuous symmetries arise commonly in nature and often give rise to coherent spatio-temporal patterns. However, because of their random locations, these patterns are not well captured by current order…