Related papers: Machine-Learned Leftmost Hessian Eigenvectors for …

Hessian-free training has become a popular parallel second or- der optimization technique for Deep Neural Network training. This study aims at speeding up Hessian-free training, both by means of decreasing the amount of data used for…

Integrating machine learning into reactive chemistry, materials discovery, and drug design is revolutionizing the development of novel molecules and materials. Machine Learning Interatomic Potentials (MLIPs) accurately predict energies and…

Established methods for unsupervised representation learning such as variational autoencoders produce none or poorly calibrated uncertainty estimates making it difficult to evaluate if learned representations are stable and reliable. In…

Stochastic neighbor embedding (SNE) and related nonlinear manifold learning algorithms achieve high-quality low-dimensional representations of similarity data, but are notoriously slow to train. We propose a generic formulation of embedding…

Multitask learning (MTL) can utilize the relatedness between multiple tasks for performance improvement. The advent of multimodal data allows tasks to be referenced by multiple indices. High-order tensors are capable of providing efficient…

To dynamically traverse challenging terrain, legged robots need to continually perceive and reason about upcoming features, adjust the locations and timings of future footfalls and leverage momentum strategically. We present a pipeline that…

Proper optimization of deep neural networks is an open research question since an optimal procedure to change the learning rate throughout training is still unknown. Manually defining a learning rate schedule involves troublesome…

We address the challenge of performing Targeted Maximum Likelihood Estimation (TMLE) after an initial Highly Adaptive Lasso (HAL) fit. Existing approaches that utilize the data-adaptive working model selected by HAL-such as the relaxed HAL…

Full-field traffic state information (i.e., flow, speed, and density) is critical for the successful operation of Intelligent Transportation Systems (ITS) on freeways. However, incomplete traffic information tends to be directly collected…

Recent years have seen a growing interest in understanding acceleration methods through the lens of ordinary differential equations (ODEs). Despite the theoretical advancements, translating the rapid convergence observed in continuous-time…

We provide an exact analysis of a class of randomized algorithms for solving overdetermined least-squares problems. We consider first-order methods, where the gradients are pre-conditioned by an approximation of the Hessian, based on a…

This paper introduces a novel learning-based Stochastic Hybrid System (LSHS) approach for detecting and classifying various contingencies in modern power systems. Specifically, the proposed method is capable of identifying hidden…

Understanding quantum phase transitions in physical systems is fundamental to characterize their behavior at low temperatures. Achieving this requires both accessing good approximations to the ground state and identifying order parameters…

Traditional maximum entropy and sparsity-based algorithms for analytic continuation often suffer from the ill-posed kernel matrix or demand tremendous computation time for parameter tuning. Here we propose a neural network method by convex…

Machine learning (ML) problems are often posed as highly nonlinear and nonconvex unconstrained optimization problems. Methods for solving ML problems based on stochastic gradient descent are easily scaled for very large problems but may…

Optimizing tensor networks with standard first-order methods often leads to slow convergence and entrapment in local minima. Although second-order optimization offers enhanced robustness, explicitly constructing the full Hessian matrix is…

The design of high-entropy alloys (HEA) with desired properties is challenging due to their large compositional space. While various machine learning (ML) models can predict specific HEA solid-solution phases (SS), predicting high-entropy…

Optimization in Deep Learning is mainly dominated by first-order methods which are built around the central concept of backpropagation. Second-order optimization methods, which take into account the second-order derivatives are far less…

Recent advances in Bayesian learning with large-scale data have witnessed emergence of stochastic gradient MCMC algorithms (SG-MCMC), such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian MCMC (SGHMC), and…

It is well known that numerical simulations of high-speed reacting flows, in the framework of state-to-state formulations, are the most detailed but also often prohibitively computationally expensive. In this work, we start to investigate…