Related papers: Relevance in the Renormalization Group and in Info…
Concept Bottleneck Models (CBMs) are regarded as inherently interpretable because they first predict a set of human-defined concepts which are used to predict a task label. For inherent interpretability to be fully realised, and ensure…
Some recent results showed that renormalization group can be considered as a promising framework to address open issues in data analysis. In this work, we focus on one of these aspects, closely related to principal component analysis for…
We describe how to evaluate approximately various physical interesting quantities in random Ising systems by direct renormalization of a finite system. The renormalization procedure is used to reduce the number of degrees of freedom to a…
The information bottleneck (IB) method aims to find compressed representations of a variable $X$ that retain the most relevant information about a target variable $Y$. We show that for a wide family of distributions -- namely, when $Y$ is…
Compact neural networks are essential for affordable and power efficient deep learning solutions. Binary Neural Networks (BNNs) take compactification to the extreme by constraining both weights and activations to two levels, $\{+1, -1\}$.…
In effective field theories, the concept of renormalization of perturbative divergences is replaced by renormalization group concepts such as relevance and universality. Universality is related to cutoff scheme independence in…
Leveraging high-quality joint representations from multimodal data can greatly enhance model performance in various machine-learning based applications. Recent multimodal learning methods, based on the multimodal information bottleneck…
In the field of compressed sensing, a key problem remains open: to explicitly construct matrices with the restricted isometry property (RIP) whose performance rivals those generated using random matrix theory. In short, RIP involves…
Interpretable Machine Learning (IML) methods are used to gain insight into the relevance of a feature of interest for the performance of a model. Commonly used IML methods differ in whether they consider features of interest in isolation,…
We view the Information Bottleneck Principle (IBP: Tishby et al., 1999; Schwartz-Ziv and Tishby, 2017) and Predictive Information Bottleneck Principle (PIBP: Still et al., 2007; Alemi, 2019) as special cases of a family of general…
Despite it being the cornerstone of BPE, the most common tokenization algorithm, the importance of compression in the tokenization process is still unclear. In this paper, we argue for the theoretical importance of compression, that can be…
This work continues the study of the relationship between sample compression schemes and statistical learning, which has been mostly investigated within the framework of binary classification. The central theme of this work is establishing…
What if Information Retrieval (IR) systems did not just retrieve relevant information that is stored in their indices, but could also "understand" it and synthesise it into a single document? We present a preliminary study that makes a…
A perturbative renormalization group is formulated for the study of Hamiltonian light-front field theory near a critical Gaussian fixed point. The only light-front renormalization group transformations found that can be approximated by…
Deep neural networks (DNNs) have achieved significant success in various applications with large-scale and balanced data. However, data in real-world visual recognition are usually long-tailed, bringing challenges to efficient training and…
We study elastic systems such as interfaces or lattices, pinned by quenched disorder. To escape triviality as a result of ``dimensional reduction'', we use the functional renormalization group. Difficulties arise in the calculation of the…
Lossy image compression networks aim to minimize the latent entropy of images while adhering to specific distortion constraints. However, optimizing the neural network can be challenging due to its nature of learning quantized latent…
The structure of the renormalization group equations for the low energy effective theory of gravity coupled to a scalar field is presented. An approximate solution to these equations with a finite number of independent renormalized…
I show that under certain conditions it is possible to define consistent irrelevant deformations of interacting conformal field theories. The deformations are finite or have a unique running scale ("quasi-finite"). They are made of an…
Interpretability is often pointed out as a key requirement for trustworthy machine learning. However, learning and releasing models that are inherently interpretable leaks information regarding the underlying training data. As such…