Related papers: Automatic Differentiable Numerical Renormalization…
The performance of deep neural networks is well-known to be sensitive to the setting of their hyperparameters. Recent advances in reverse-mode automatic differentiation allow for optimizing hyperparameters with gradients. The standard way…
Natural gradient descent (NGD) is a powerful optimization technique for machine learning, but the computational complexity of the inverse Fisher information matrix limits its application in training deep neural networks. To overcome this…
Self-paced learning and hard example mining re-weight training instances to improve learning accuracy. This paper presents two improved alternatives based on lightweight estimates of sample uncertainty in stochastic gradient descent (SGD):…
We apply the density matrix renormalization group (DMRG) method to a non-equilibrium problem: the asymmetric exclusion process in one dimension. We study the stationary state of the process to calculate the particle density profile…
We consider the problem of denoising with the help of prior information taken from a database of clean signals or images. Denoising with variational methods is very efficient if a regularizer well adapted to the nature of the data is…
In this paper, we introduce a physics-driven regularization method for training of deep neural networks (DNNs) for use in engineering design and analysis problems. In particular, we focus on prediction of a physical system, for which in…
Recent work has shown that forward- and reverse- mode automatic differentiation (AD) over the reals is almost always correct in a mathematically precise sense. However, actual programs work with machine-representable numbers (e.g.,…
Recurrent neural networks are a successful neural architecture for many time-dependent problems, including time series analysis, forecasting, and modeling of dynamical systems. Training such networks with backpropagation through time is a…
The recursive Newton-Euler Algorithm (RNEA) is a popular technique for computing the dynamics of robots. RNEA can be framed as a differentiable computational graph, enabling the dynamics parameters of the robot to be learned from data via…
We develop a renormalization group (RG) procedure that includes important system-specific features. The key ingredient is to systematize the coarse graining procedure that generates the RG flow. The coarse graining technology comes from…
Recently continuous relaxations have been proposed in order to learn Directed Acyclic Graphs (DAGs) from data by backpropagation, instead of using combinatorial optimization. However, a number of techniques for fully discrete…
We introduce RGFlow, a deep neural network-based real-space renormalization group (RG) framework tailored for continuum scalar field theories. Leveraging generative capabilities of flow-based neural networks, RGFlow autonomously learns…
We show how the density-matrix numerical renormalization group (DM-NRG) method can be used in combination with non-Abelian symmetries such as SU(N), where the decomposition of the direct product of two irreducible representations requires…
Vanishing (and exploding) gradients effect is a common problem for recurrent neural networks with nonlinear activation functions which use backpropagation method for calculation of derivatives. Deep feedforward neural networks with many…
The development of tensor renormalization group (TRG) algorithm in higher dimensions is an important and urgent task, as the TRG is expected to provide a way to overcome the sign problem in lattice quantum chromodynamics (QCD) calculations…
The application of Wilson's Numerical Renormalization Group (NRG) method to dissipative quantum impurity models, in particular the sub-ohmic spin-boson model, has led to conclusions regarding the quantum critical behavior which are in…
This work establishes that a physical system can perform statistical learning without gradient computations, via an Agnostic Equilibrium Propagation (Aeqprop) procedure that combines energy minimization, homeostatic control, and nudging…
In this paper, we focus on the interpretability of deep neural network. Our work is motivated by the renormalization group (RG) in statistical mechanics. RG plays the role of a bridge connecting microscopical properties and macroscopic…
It is shown that the renormalization group (RG) method for global analysis can be formulated in the context of the classical theory of envelopes: Several examples from partial differential equations are analyzed. The amplitude equations…
Learning dynamical systems that respect physical symmetries and constraints remains a fundamental challenge in data-driven modeling. Integrating physical laws with graph neural networks facilitates principled modeling of complex N-body…