Related papers: KO: Kinetics-inspired Neural Optimizer with PDE Si…

Optimization is often cast as a deterministic problem, where the solution is found through some iterative procedure such as gradient descent. However, when training neural networks the loss function changes over (iteration) time due to the…

This research delves into optimizing mechanism design, with an emphasis on the energy efficiency and the expansive design possibilities of reciprocating mechanisms. It investigates how to efficiently integrate Computer-Aided Design (CAD)…

Neuroscientists fit morphologically and biophysically detailed neuron simulations to physiological data, often using evolutionary algorithms. However, such gradient-free approaches are computationally expensive, making convergence slow when…

Combinatorial optimization problems are notoriously challenging for neural networks, especially in the absence of labeled instances. This work proposes an unsupervised learning framework for CO problems on graphs that can provide integral…

We introduce Velocity-Regularized Adam (VRAdam), a physics-inspired optimizer for training deep neural networks that draws on ideas from quartic terms for kinetic energy with its stabilizing effects on various system dynamics. Previous…

The inherent slow imaging speed of Magnetic Resonance Image (MRI) has spurred the development of various acceleration methods, typically through heuristically undersampling the MRI measurement domain known as k-space. Recently, deep neural…

This project focuses on optimizing input parameters of a partial derivative function of a fine model using Neural network-based Space Mapping Optimization (SMO). The fine model is known for its high accuracy but is computationally…

A general framework of unsupervised learning for combinatorial optimization (CO) is to train a neural network (NN) whose output gives a problem solution by directly optimizing the CO objective. Albeit with some advantages over traditional…

Koopman operator theory, a powerful framework for discovering the underlying dynamics of nonlinear dynamical systems, was recently shown to be intimately connected with neural network training. In this work, we take the first steps in…

Gradient-based first-order adaptive optimization methods such as the Adam optimizer are prevalent in training artificial networks, achieving the state-of-the-art results. This work attempts to answer the question whether it is viable for…

Relevant combinatorial optimization problems (COPs) are often NP-hard. While they have been tackled mainly via handcrafted heuristics in the past, advances in neural networks have motivated the development of general methods to learn…

Iterative optimization is central to modern artificial intelligence (AI) and provides a crucial framework for understanding adaptive systems. This review provides a unified perspective on this subject, bridging classic theory with neural…

PDE-Constrained Optimization (PDECO) problems can be accelerated significantly by employing gradient-based methods with surrogate models like neural operators compared to traditional numerical solvers. However, this approach faces two key…

Metaheuristic algorithms are optimization methods that are inspired by real phenomena in nature or the behavior of living beings, e.g., animals, to be used for solving complex problems, as in engineering, energy optimization, health care,…

We propose a novel Bayesian optimization (BO) procedure aimed at identifying the ``profile optima'' of a deterministic black-box computer simulation that has a single control parameter and multiple nuisance parameters. The profile optima…

The optimization algorithm and its hyperparameters can significantly affect the training speed and resulting model accuracy in machine learning applications. The wish list for an ideal optimizer includes fast and smooth convergence to low…

We investigate the potential of numerical algorithms to decipher the kinetic parameters involved in multi-step chemical reactions. To this end we study a dimerization kinetics of protein as a model system. We follow the dimerization…

In this work, we study physics-informed neural networks (PINNs) constrained by partial differential equations (PDEs) and their application in approximating PDEs with two characteristic scales. From a continuous perspective, our formulation…

Neural Combinatorial Optimization (NCO) has emerged as a promising approach for NP-hard problems. However, prevailing RL-based methods suffer from low sample efficiency due to sparse rewards and underused solutions. We propose Best-anchored…

We calibrate parameters of neural networks that model forces in interaction dynamics with the help of the Consensus-based global optimization method (CBO). We state the general framework of interaction particle systems driven by neural…