Related papers: Towards quantifying information flows: relative en…
The contact process and the slightly different susceptible-infected-susceptible model are studied on long-range connected networks in the presence of random transition rates by means of a strong disorder renormalization group method and…
We explore entanglement loss along renormalization group trajectories as a basic quantum information property underlying their irreversibility. This analysis is carried out for the quantum Ising chain as a transverse magnetic field is…
The Renormalisation Group (RG) provides a framework in which it is possible to assess whether a deep-learning network is sensitive to small changes in the input data and hence prone to error, or susceptible to adversarial attack. Distinct…
We investigate the second-order R\'enyi entanglement entropy at the quantum critical point of a spin-1/2 antiferromagnetic Heisenberg model on a columnar dimerized square lattice. The universal constant $\gamma$ in the area-law scaling…
We apply real-space RG methods to study two quantum group invariant Hamiltonians, that of the XXZ model and the Ising model in a transverse field defined in an open chain with appropiate boundary terms. The quantum group symmetry is…
This paper establishes risk convergence and asymptotic weight matrix alignment --- a form of implicit regularization --- of gradient flow and gradient descent when applied to deep linear networks on linearly separable data. In more detail,…
We establish an exact equivalence between the Functional Renormalization Group (FRG) and the Ricci flow modified by a potential-driven diffeomorphism. By reformulating the Polchinski exact renormalization group equation into an…
Reinforcement learning (RL) has become a key approach for enhancing reasoning in large language models (LLMs), yet scalable training is often hindered by the rapid collapse of policy entropy, which leads to premature convergence and…
The information bottleneck (IB) principle has been suggested as a way to analyze deep neural networks. The learning dynamics are studied by inspecting the mutual information (MI) between the hidden layers and the input and output. Notably,…
Recently, renormalized entropy was proposed as a novel measure of relative entropy (P. Saparin et al., Chaos, Solitons & Fractals 4, 1907 (1994)) and applied to several physiological time sequences, including EEGs of patients with epilepsy.…
We use a mix of field theoretic and holographic techniques to elucidate various properties of quantum entanglement entropy. In (3+1)-dimensional conformal field theory we study the divergent terms in the entropy when the entangling surface…
Guided by critical systems found in nature we develop a novel mechanism consisting of inhomogeneous polynomial regularisation via which we can induce scale invariance in deep learning systems. Technically, we map our deep learning (DL)…
Relative entropy serves as a cornerstone concept in quantum information theory. In this work, we study relative entropy of random states from major generic state models of Hilbert-Schmidt and Bures-Hall ensembles. In particular, we derive…
The random matrix ensembles (RME) of quantum statistical Hamiltonians, e.g. Gaussian random matrix ensembles (GRME) and Ginibre random matrix ensembles (Ginibre RME), are applied in literature to following quantum statistical systems:…
We examine non-relativistic holographic RG flows by working with Einstein-Maxwell-scalar theories which support geometries that break Lorentz invariance at some energy scale. We adopt the superpotential formalism, which helps us…
Recurrent neural networks (RNNs) are complex dynamical systems, capable of ongoing activity without any driving input. The long-term behavior of free-running RNNs, described by periodic, chaotic and fixed point attractors, is controlled by…
This book develops an effective theory approach to understanding deep neural networks of practical relevance. Beginning from a first-principles component-level picture of networks, we explain how to determine an accurate description of the…
The aim of this paper is to provide new theoretical and computational understanding on two loss regularizations employed in deep learning, known as local entropy and heat regularization. For both regularized losses we introduce variational…
We propose that learning in deep neural networks proceeds in two phases: a rapid curve fitting phase followed by a slower compression or coarse graining phase. This view is supported by the shared temporal structure of three phenomena:…
The analysis of complex physical systems hinges on the ability to extract the relevant degrees of freedom from among the many others. Though much hope is placed in machine learning, it also brings challenges, chief of which is…