Related papers: Towards quantifying information flows: relative en…
The Neural Network Field Theory correspondence (NNFT) is a mapping from neural network (NN) architectures into the space of statistical field theories (SFTs). The Bayesian renormalization group (BRG) is an information-theoretic coarse…
We study regularization for the deep linear network (DLN) using the entropy formula introduced in arXiv:2509.09088. The equilibria and gradient flow of the free energy on the Riemannian manifold of end-to-end maps of the DLN are…
Hierarchical structures, which include multiple levels, are prevalent in statistical and machine-learning models as well as physical systems. Extending the foundational result that the maximum entropy distribution under mean constraints is…
We show with several examples that renormalization group (RG) theory can be used to understand singular and reductive perturbation methods in a unified fashion. Amplitude equations describing slow motion dynamics in nonequilibrium phenomena…
Recently, information theoretic analysis has become a popular framework for understanding the generalization behavior of deep neural networks. It allows a direct analysis for stochastic gradient/Langevin descent (SGD/SGLD) learning…
We analyze the effect of decoherence, modelled by local quantum channels, on quantum critical states and we find universal properties of the resulting mixed state's entanglement, both between system and environment and within the system.…
In this paper, we present a new approach to interpret deep learning models. By coupling mutual information with network science, we explore how information flows through feedforward networks. We show that efficiently approximating mutual…
We introduce a "renormalized entanglement entropy" which is intrinsically UV finite and is most sensitive to the degrees of freedom at the scale of the size R of the entangled region. We illustrated the power of this construction by showing…
It has been shown \citep{broeck90:physicalreview,patarnello87:europhys} that feedforward Boolean networks can learn to perform specific simple tasks and generalize well if only a subset of the learning examples is provided for learning.…
Here, the entanglement entropy is calculated at the quantum multicritical point of the random transverse-field Ising model (RTIM). We use an efficient implementation of the strong disorder renormalization group method in two and three…
Four decades after its first postulation by Bekenstein, black hole entropy remains mysterious. It has long been suggested that the entanglement entropy of quantum fields on the black hole gravitational background should represent at least…
We study the critical properties of the weakly disordered two-dimensional Ising and Baxter models in terms of the renormalization group (RG) theory generalized to take into account the replica symmetry breaking (RSB) effects. Recently it…
We introduce hardness in relative entropy, a new notion of hardness for search problems which on the one hand is satisfied by all one-way functions and on the other hand implies both next-block pseudoentropy and inaccessible entropy, two…
Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building…
The random matrix ensembles (RME), especially Gaussian random matrix ensembles GRME and Ginibre random matrix ensembles, are applied to following quantum systems: nuclear systems, molecular systems, and two-dimensional electron systems…
Random matrix ensembles (RME) of quantum statistical Hamiltonian operators, {\em e.g.} Gaussian random matrix ensembles (GRME) and Ginibre random matrix ensembles (Ginibre RME), found applications in literature in study of following quantum…
We explore how minimal neural networks can invert the renormalization group (RG) coarse-graining procedure in the two-dimensional Ising model, effectively ``dreaming up'' microscopic configurations from coarse-grained states. This task -…
Recently, a rigorous yet concise formula has been derived to evaluate the information flow, and hence the causality in a quantitative sense, between time series. To assess the importance of a resulting causality, it needs to be normalized.…
In this Thesis, several results in quantum information theory are collected, most of which use entropy as the main mathematical tool. *While a direct generalization of the Shannon entropy to density matrices, the von Neumann entropy behaves…
A method is provided for designing and training noise-driven recurrent neural networks as models of stochastic processes. The method unifies and generalizes two known separate modeling approaches, Echo State Networks (ESN) and Linear…