Related papers: Batched Second-Order Adjoint Sensitivity for Reduc…

We present a new accelerated stochastic second-order method that is robust to both gradient and Hessian inexactness, which occurs typically in machine learning. We establish theoretical lower bounds and prove that our algorithm achieves…

Neural differential equations may be trained by backpropagating gradients via the adjoint method, which is another differential equation typically solved using an adaptive-step-size numerical differential equation solver. A proposed step is…

Reduced-order modeling lies at the interface of numerical analysis and data-driven scientific computing, providing principled ways to compress high-fidelity simulations in science and engineering. We propose a training framework that…

We propose an algorithm for the computational homogenization of locally periodic hyperelastic structures undergoing large deformations due to external quasi-static loading. The algorithm performs clustering of macroscopic deformations into…

Large number of weights in deep neural networks makes the models difficult to be deployed in low memory environments such as, mobile phones, IOT edge devices as well as "inferencing as a service" environments on cloud. Prior work has…

The dynamic behavior of jointed assemblies exhibiting friction nonlinearities features amplitude-dependent dissipation and stiffness. To develop numerical simulations for predictive and design purposes, macro-scale High Fidelity Models…

This work presents an illustrative application of the Second-Order Adjoint Sensitivity Analysis Methodology (2nd-ASAM) developed by Cacuci (2015) to a paradigm nonlinear heat conduction benchmark, which models a conceptual experimental test…

This paper considers distributed optimization problems, where each agent cooperatively minimizes the sum of local objective functions through the communication with its neighbors. The widely adopted distributed gradient method in solving…

Nonconvex optimization problems are widespread in modern machine learning and data science. We introduce an extrapolation strategy into a class of preconditioned second-order convex splitting algorithms for nonconvex optimization problems.…

This work presents the Second-Order Sensitivity Analysis Methodology (2nd-ASAM) for nonlinear systems. This methodology yields exactly and efficiently the second-order functional derivatives of system responses (associated with physical,…

A range of optimization cases of two-dimensional Stefan problems, solved using a tracking-type cost-functional, is presented. A level set method is used to capture the interface between the liquid and solid phases and an immersed boundary…

In this paper, we apply the practical GADI-HS iteration as a smoother in algebraic multigrid (AMG) method for solving second-order non-selfadjoint elliptic problem. Additionally, we prove the convergence of the derived algorithm and…

3D Gaussian Splatting (3DGS) is widely used for novel view synthesis due to its high rendering quality and fast inference time. However, 3DGS predominantly relies on first-order optimizers such as Adam, which leads to long training times.…

In gradient-based time domain topology optimization, design sensitivity analysis (DSA) of the dynamic response is essential, and requires high computational cost to directly differentiate, especially for high-order dynamic system. To…

This paper introduces a second-order convex splitting scheme for gradient flows arising in phase-field models, based on the backward differentiation formula (BDF2) for the implicit part and the Adams-Bashforth method for the nonlinear and…

We present an efficient distributed memory parallel algorithm for computing connected components in undirected graphs based on Shiloach-Vishkin's PRAM approach. We discuss multiple optimization techniques that reduce communication volume as…

This paper proposes a new approach for the calibration of material parameters in local elastoplastic constitutive models. The calibration is posed as a constrained optimization problem, where the constitutive model evolution equations for a…

Large scale optimization problems are ubiquitous in machine learning and data analysis and there is a plethora of algorithms for solving such problems. Many of these algorithms employ sub-sampling, as a way to either speed up the…

Advanced optimization algorithms such as Newton method and AdaGrad benefit from second order derivative or second order statistics to achieve better descent directions and faster convergence rates. At their heart, such algorithms need to…

The identification of primal variables and adjoint variables is usually done via indices in operator overloading algorithmic differentiation tools. One approach is a linear management scheme, which is easy to implement and supports memory…