Related papers: On Triangular Inequality of the Discounted Least I…
We provide a spectrum of new theoretical insights and practical results for finding a Minimum Dilation Triangulation (MDT), a natural geometric optimization problem of considerable previous attention: Given a set $P$ of $n$ points in the…
We introduce an information-theoretic quantity with similar properties to mutual information that can be estimated from data without making explicit assumptions on the underlying distribution. This quantity is based on a recently proposed…
The idea of slicing divergences has been proven to be successful when comparing two probability measures in various machine learning applications including generative modeling, and consists in computing the expected value of a `base…
While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, up to now Shannon's entropy power inequality (EPI) is an exception: Existing information theoretic proofs of the…
Decomposition analysis is a critical tool for understanding the social and spatial dimensions of segregation and diversity. In this paper, I highlight the conceptual, mathematical, and empirical distinctions between segregation and…
Empirically, Deep Learning (DL) has demonstrated unprecedented success in practical applications. However, DL remains by and large a mysterious "black-box", spurring recent theoretical research to build its mathematical foundations. In this…
In a paper [8] the authors classify entropy into three categories, as a thermodynamics quantity, as a measure of information production, and as a means of statistical inference. An entropy measure introduced by Mathai falls into the second…
We present ITE (information theoretical estimators) a free and open source, multi-platform, Matlab/Octave toolbox that is capable of estimating many different variants of entropy, mutual information, divergence, association measures, cross…
Relative entropy is a powerful measure of the dissimilarity between two statistical field theories in the continuum. In this work, we study the relative entropy between Gaussian scalar field theories in a finite volume with different masses…
We consider high dimensional Wishart matrices $\mathbb{X} \mathbb{X}^{\top}$ where the entries of $\mathbb{X} \in {\mathbb{R}^{n \times d}}$ are i.i.d. from a log-concave distribution. We prove an information theoretic phase transition:…
This archiving article consists of several short reports on the discussions between the two authors over the past two years at Oxford and Madrid, and their work carried out during that period on the upper bound of the Kullback-Leibler…
Defining similarity is a fundamental challenge in information science. Watanabe's Ugly Duckling Theorem highlights diversity, while algorithmic information theory emphasizes depth through Information Distance. We propose a…
We study a curve of Gibbsian families of complex 3x3-matrices and point out new features, absent in commutative finite-dimensional algebras: a discontinuous maximum-entropy inference, a discontinuous entropy distance and non-exposed faces…
We present an information-theoretic assessment of atomic and molecular densities in the ground state and under a range of physical scenarios--excitation, confinement, and ensemblization. Comparisons across densities obtained from…
Gathering the most information by picking the least amount of data is a common task in experimental design or when exploring an unknown environment in reinforcement learning and robotics. A widely used measure for quantifying the…
We consider a family of prior probability distributions of particular interest, all being defined on the three-dimensional convex set of two-level quantum systems. Each distribution is, following recent work of Petz and Sudar, taken to be…
Information theory has been taken as a prospective tool for quantifying the complexity of complex networks. In this paper, we first study the information entropy or uncertainty of a path using the information theory. Then we apply the path…
Quantifying the directionality of information flow is instrumental in understanding, and possibly controlling, the operation of many complex systems, such as transportation, social, neural, or gene-regulatory networks. The standard Transfer…
The data processing inequality (DPI) is a fundamental feature of information theory. Informally it states that you cannot increase the information content of a quantum system by acting on it with a local physical operation. When the smooth…
This paper introduces a comprehensive framework for complex-valued probability measures and explores their novel applications in information theory and statistical analysis. We define a complex probability measure as a phase-modulated…