Related papers: Criticality & Deep Learning II: Momentum Renormali…
In the framework of the renormalization-group (RG) approach, critical phenomena can be investigated by studying the RG flow of multi-parameter $\Phi^4$ field theories with an $N$-component fundamental field, containing up to 4th-order…
We study the role of $L_2$ regularization in deep learning, and uncover simple relations between the performance of the model, the $L_2$ coefficient, the learning rate, and the number of training steps. These empirical relations hold when…
We present a variational renormalization group (RG) approach using a deep generative model based on normalizing flows. The model performs hierarchical change-of-variables transformations from the physical space to a latent space with…
In an attempt to better understand generalization in deep learning, we study several possible explanations. We show that implicit regularization induced by the optimization method is playing a key role in generalization and success of deep…
We suggest a new, renormalization group (RG) based, nonperturbative method for treating the intermittency problem of fully developed turbulence which also includes the effects of a finite boundary of the turbulent flow. The key idea is not…
We present a novel approach (DyNODE) that captures the underlying dynamics of a system by incorporating control in a neural ordinary differential equation framework. We conduct a systematic evaluation and comparison of our method and…
The density matrix renormalization group (DMRG) is applied to some one-dimensional reaction-diffusion models in the vicinity of and at their critical point. The stochastic time evolution for these models is given in terms of a non-symmetric…
Generic deep learning (DL) networks for image restoration like denoising and interpolation lack mathematical interpretability, require voluminous training data to tune a large parameter set, and are fragile in the face of covariate shift.…
In this paper we explore the relation between distributionally robust learning and different forms of regularization to enforce robustness of deep neural networks. In particular, starting from a concrete min-max distributionally robust…
Dynamical models identified from data are frequently employed in control system design. However, decoupling system identification from controller synthesis can result in situations where no suitable controller exists after a model has been…
Critical behaviour of a fluid, subjected to strongly anisotropic turbulent mixing, is studied by means of the field theoretic renormalization group. As a simplified model, relaxational stochastic dynamics of a non-conserved scalar order…
We show with several examples that renormalization group (RG) theory can be used to understand singular and reductive perturbation methods in a unified fashion. Amplitude equations describing slow motion dynamics in nonequilibrium phenomena…
Deep reinforcement learning (RL) agents often fail to generalize to unseen environments (yet semantically similar to trained agents), particularly when they are trained on high-dimensional state spaces, such as images. In this paper, we…
In many machine learning problems the output should not depend on the order of the input. Such "permutation invariant" functions have been studied extensively recently. Here we argue that temporal architectures such as RNNs are highly…
We present a renormalization group (RG) procedure which works naturally on a wide class of interacting one-dimension models based on perturbed (possibly strongly) continuum conformal and integrable models. This procedure integrates Kenneth…
We present a simple method, combining the density-matrix renormalization-group (DMRG) algorithm with finite-size scaling, which permits the study of critical behavior in quantum spin chains. Spin moments and dimerization are induced by…
In physics one attempts to infer the rules governing a system given only the results of imperfect measurements. Hence, microscopic theories may be effectively indistinguishable experimentally. We develop an operationally motivated procedure…
The role of $L^2$ regularization, in the specific case of deep neural networks rather than more traditional machine learning models, is still not fully elucidated. We hypothesize that this complex interplay is due to the combination of…
Reinforcement Learning (RL) of robotic manipulation skills, despite its impressive successes, stands to benefit from incorporating domain knowledge from control theory. One of the most important properties that is of interest is control…
Data assisted reconstruction algorithms, incorporating trained neural networks, are a novel paradigm for solving inverse problems. One approach is to first apply a classical reconstruction method and then apply a neural network to improve…