相关论文: Mutual Information of Generalized Free Fields

Estimating and optimizing Mutual Information (MI) is core to many problems in machine learning; however, bounding MI in high dimensions is challenging. To establish tractable and scalable objectives, recent work has turned to variational…

The holographic mutual information for the small separation of two circles and two strips in 2+1 dimensional space-time is considered based on the known exact minimal surfaces spanning the boundaries on AdS4. The results suggest a…

The Generalized Degrees of Freedom (GDoF) of the two user interference channel are characterized for all parameter regimes under the assumption of finite precision channel state information at the transmitters (CSIT), when a limited amount…

We discuss upper bounds on the mutual information for disjoint spherical regions of the CFT vacuum. To prove our bounds, we utilize the modular nuclearity condition, which is in turn related to finiteness of the thermal partition function…

I present several new relations between mutual information (MI) and statistical estimation error for a system that can be regarded simultaneously as a communication channel and as an estimator of an input parameter. I first derive a…

Recently, Mutual Information (MI) has attracted attention in bounding the generalization error of Deep Neural Networks (DNNs). However, it is intractable to accurately estimate the MI in DNNs, thus most previous works have to relax the MI…

For several styles of fidelity constraints -- guaranteed distortion, conditional excess distortion, excess distortion -- we show mutual information upper bounds on the minimum expected description length needed to represent a random…

Understanding the generalization abilities of modern machine learning algorithms has been a major research topic over the past decades. In recent years, the learning dynamics of Stochastic Gradient Descent (SGD) have been related to…

The existence of an exactly marginal deformation in a conformal field theory is very special, but it is not well understood how this is reflected in the allowed dimensions and OPE coefficients of local operators. To shed light on this…

The basic methods of constructing the sets of mutually unbiased bases in the Hilbert space of an arbitrary finite dimension are discussed and an emerging link between them is outlined. It is shown that these methods employ a wide range of…

In the federated learning system, parameter gradients are shared among participants and the central modulator, while the original data never leave their protected source domain. However, the gradient itself might carry enough information…

Data augmentation is one of the most widely used techniques to improve generalization in modern machine learning, often justified by its ability to promote invariance to label-irrelevant transformations. However, its theoretical role…

This paper presents a novel approach to machine learning algorithm design based on information theory, specifically mutual information (MI). We propose a framework for learning and representing functional relationships in data using…

Based on results of E. DiBenedetto and D. Hoff we propose an explicit finite difference scheme for the one dimensional Generalized Porous Medium Equation $\partial_t u=\partial_{xx}^2 \Phi(u)$. The scheme allows to track the moving free…

Recent work has established that the conditional mutual information (CMI) framework of Steinke and Zakynthinou (2020) is expressive enough to capture generalization guarantees in terms of algorithmic stability, VC dimension, and related…

Quantifying degrees of fusion and separability between data groups in representation space is a fundamental problem in representation learning, particularly under domain shift. A meaningful metric should capture fusion-altering factors like…

Text-to-image generation and image captioning are recently emerged as a new experimental paradigm to assess machine intelligence. They predict continuous quantity accompanied by their sampling techniques in the generation, making evaluation…

Estimating mutual information (MI) from samples is a fundamental problem in statistics, machine learning, and data analysis. Recently it was shown that a popular class of non-parametric MI estimators perform very poorly for strongly…

We define the lower and upper mutual dimensions $mdim(x:y)$ and $Mdim(x:y)$ between any two points $x$ and $y$ in Euclidean space. Intuitively these are the lower and upper densities of the algorithmic information shared by $x$ and $y$. We…

In this paper, we introduce and investigate the notions of Mean Dimension and Metric Mean Dimension for generalized iterated function systems (IFS). We establish basic properties of these invariants and prove that Mean Dimension is always…