相关论文: Relevance in the Renormalization Group and in Info…

The training dynamics of hidden layers in deep learning are poorly understood in theory. Recently, the Information Plane (IP) was proposed to analyze them, which is based on the information-theoretic concept of mutual information (MI). The…

we show there exists a mathematically consistent framework in which the Renormalization Program can be understood in a natural manner. The framework does not require any violations of mathematical rigor usually associated with the…

Complex systems are characterised by a tight, nontrivial interplay of their constituents, which gives rise to a multi-scale spectrum of emergent properties. In this scenario, it is practically and conceptually difficult to identify those…

We define the relevant information in a signal $x\in X$ as being the information that this signal provides about another signal $y\in \Y$. Examples include the information that face images provide about the names of the people portrayed, or…

Unbiased Learning to Rank (ULTR) aims to train unbiased ranking models from biased click logs, by explicitly modeling a generation process for user behavior and fitting click data based on examination hypothesis. Previous research found…

The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations $T$ of some random variable $X$ for the task of predicting $Y$. It is defined as a constrained optimization problem which maximizes…

In the realm of neural network models, the perpetual challenge remains in retaining task-relevant information while effectively discarding redundant data during propagation. In this paper, we introduce IB-AdCSCNet, a deep learning model…

Split learning is a privacy-preserving distributed learning paradigm in which an ML model (e.g., a neural network) is split into two parts (i.e., an encoder and a decoder). The encoder shares so-called latent representation, rather than raw…

Decision-making in complex systems often relies on machine learning models, yet highly accurate models such as XGBoost and neural networks can obscure the reasoning behind their predictions. In operations research applications,…

Being able to interpret, or explain, the predictions made by a machine learning model is of fundamental importance. This is especially true when there is interest in deploying data-driven models to make high-stakes decisions, e.g. in…

In physics we often use very simple models to describe systems with many degrees of freedom, but it is not clear why or how this success can be transferred to the more complex biological context. We consider models for the joint…

We study the critical behaviour of symmetric $\phi^4_4$ theory including irrelevant terms of the form $\phi^{4+2n}/\Lambda_0^{2n}$ in the bare action, where $\Lambda_0$ is the UV cutoff (corresponding e.g. to the inverse lattice spacing for…

Learned representations at the level of characters, sub-words, words and sentences, have each contributed to advances in understanding different NLP tasks and linguistic phenomena. However, learning textual embeddings is costly as they are…

The general structure of the renormalization group equations for the low energy effective field theory formulation of pure gravity is presented. The solution of these equations takes a particular simple form if the mass scale of the…

Topology and many-body localization (MBL) have opened new avenues for preserving quantum information at finite energy density. Resonant delocalization plays a crucial role in destabilizing these phenomena. In this work, we study the…

In this paper we apply the previously introduced approximation method based on the ANOVA (analysis of variance) decomposition and Grouped Transformations to synthetic and real data. The advantage of this method is the interpretability of…

We present a physical interpretation of machine learning functions, opening up the possibility to control properties of statistical systems via the inclusion of these functions in Hamiltonians. In particular, we include the predictive…

In recent several years, the information bottleneck (IB) principle provides an information-theoretic framework for deep multi-view clustering (MVC) by compressing multi-view observations while preserving the relevant information of multiple…

We propose a new constraint on the structure of strongly coupled field theories. The constraint takes the form of an inequality limiting the number of degrees of freedom in the infrared description of a theory relative to the number of…

Connecting orbits are important invariant structures in the state space of nonlinear systems and various techniques are designed for their computation. However, a uniform analytic approximation of the whole orbit seems rare. Here, based on…