Related papers: Towards quantifying information flows: relative en…
We explore a large class of correlation measures called the $\alpha-z$ R\'enyi mutual informations (RMIs). Unlike the commonly used notion of RMI involving linear combinations of R\'enyi entropies, the $\alpha-z$ RMIs are positive…
When and why representations learned by different deep neural networks are similar is an active research topic. We choose to address these questions from the perspective of identifiability theory, which suggests that a measure of…
Recurrent neural networks (RNNs) are a class of neural networks that have emerged from the paradigm of artificial intelligence and has enabled lots of interesting advances in the field of natural language processing. Interestingly, these…
Information Bottlenecks (IBs) learn representations that generalize to unseen data by information compression. However, existing IBs are practically unable to guarantee generalization in real-world scenarios due to the vacuous…
This study aims to quantify and visualize the degradation of fidelity (information degradation) that inevitably accompanies the replication of information within the framework of information thermodynamics and to propose an…
The asymptotic correspondence between the probability mass function of the $q$-deformed multinomial distribution and the $q$-generalised Kullback-Leibler divergence, also known as Tsallis relative entropy, is established. The probability…
We introduce graph normalizing flows: a new, reversible graph neural network model for prediction and generation. On supervised tasks, graph normalizing flows perform similarly to message passing neural networks, but at a significantly…
Some recent results showed that renormalization group can be considered as a promising framework to address open issues in data analysis. In this work, we focus on one of these aspects, closely related to principal component analysis for…
In the context of Wilsonian Renormalization, renormalization group (RG) flows are a set of differential equations that defines how the coupling constants of a theory depend on an energy scale. These equations closely resemble…
We analyze a semi-infinite one-dimensional random walk process with a biased motion that is incremental in one direction and long-range in the other. On a network with a fixed hierarchy of long-range jumps, we find with exact…
Inferring and comparing complex, multivariable probability density functions is fundamental to problems in several fields, including probabilistic learning, network theory, and data analysis. Classification and prediction are the two faces…
The $n$-index R\'enyi mutual information and transfer entropy for the two-dimensional kinetic Ising model with arbitrary single-spin dynamics in the thermodynamic limit are derived as functions of thermodynamic quantities. By means of Monte…
We derive constraints on renormalization group (RG) flows and stability of phases in nonequilibrium systems using quantum information inequalities. These constraints involve conditional mutual information (CMI), which quantifies…
Using holography, we study the entanglement entropy of strongly coupled field theories perturbed by operators that trigger an RG flow from a conformal field theory in the ultraviolet (UV) to a new theory in the infrared (IR). The…
Information theoretic geometry near critical points in classical and quantum systems is well understood for exactly solvable systems. Here we show that renormalization group flow equations can be used to construct the information metric and…
The integration of high-energy degrees of freedom along the renormalization group (RG) flow in Poincar\'e-invariant theories can be captured by a monotonic c-function. For such theories, holographic monotonic c-functions have been…
Lossy image compression networks aim to minimize the latent entropy of images while adhering to specific distortion constraints. However, optimizing the neural network can be challenging due to its nature of learning quantized latent…
We use the holographic method to investigate an RG flow and IR physics of a two-dimensional conformal field theory (CFT) deformed by a relevant scalar operator. On the dual gravity side, a renormalization group (RG) flow from a UV to IR CFT…
Deep learning models have proven enormously successful at using multiple layers of representation to learn relevant features of structured data. Encoding physical symmetries into these models can improve performance on difficult tasks, and…
X-ray microscopy (XRM) is commonly used to obtain three-dimensional information on internal microstructure, but the imaging pipeline introduces noise, redundancy and information loss at multiple stages. This paper treats the XRM workflow as…