Related papers: Common Information, Matroid Representation, and Se…
The problem of low-rank matrix completion with heterogeneous and sub-exponential (as opposed to homogeneous and Gaussian) noise is particularly relevant to a number of applications in modern commerce. Examples include panel sales data and…
Rough set theory is a useful tool to deal with uncertain, granular and incomplete knowledge in information systems. And it is based on equivalence relations or partitions. Matroid theory is a structure that generalizes linear independence…
In frequently repeated matching scenarios, individuals may require diversification in their choices. Therefore, when faced with a set of potential outcomes, each individual may have an ideal lottery over outcomes that represents their…
We propose an information-theoretic framework for matrix completion. The theory goes beyond the low-rank structure and applies to general matrices of "low description complexity". Specifically, we consider $m\times n$ random matrices…
Analysing how information flows along the layers of a multilayer perceptron is a topic of paramount importance in the field of artificial neural networks. After framing the problem from the point of view of information theory, in this…
Mutual information is commonly used as a measure of similarity between competing labelings of a given set of objects, for example to quantify performance in classification and community detection tasks. As argued recently, however, the…
The Ingleton inequality first appeared in matroid theory, where Ingleton proved in 1971 that every rank function coming from a representable matroid on four subsets satisfies a particular inequality. Because this inequality is not implied…
This paper is divided into two parts. In the first part, we develop a general method for expressing ranks of matrix expressions that involve Moore-Penrose inverses, group inverses, Drazin inverses, as well as weighted Moore-Penrose inverses…
Information theory plays an indispensable role in the development of algorithm-independent impossibility results, both for communication problems and for seemingly distinct areas such as statistics and machine learning. While numerous…
We systematically study how properties of abstract operator systems help classifying linear matrix inequality definitions of sets. Our main focus is on polyhedral cones, the 3-dimensional Lorentz cone, where we can completely describe all…
While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, Shannon's entropy power inequality (EPI) seems to be an exception: available information theoretic proofs of the…
We present an algebraic framework which simultaneously generalizes the notion of linear subspaces, matroids, valuated matroids, oriented matroids, and regular matroids. To do this, we first introduce algebraic objects called tracts which…
Every sufficiently big matrix with small spectral norm has a nearby low-rank matrix if the distance is measured in the maximum norm (Udell & Townsend, SIAM J Math Data Sci, 2019). We use the Hanson--Wright inequality to improve the estimate…
The matroid parity (or matroid matching) problem, introduced as a common generalization of matching and matroid intersection problems, is so general that it requires an exponential number of oracle calls. Nevertheless, Lov\'asz (1980)…
During the last two decades, concentration of measure has been a subject of various exciting developments in convex geometry, functional analysis, statistical physics, high-dimensional statistics, probability theory, information theory,…
We consider methods for aggregating preferences that are based on the resolution of discrete optimization problems. The preferences are represented by arbitrary binary relations (possibly weighted) or incomplete paired comparison matrices.…
As part of the recent developments in infinite matroid theory, there have been a number of conjectures about how standard theorems of finite matroid theory might extend to the infinite setting. These include base packing, base covering, and…
In a matroid secretary problem, one is presented with a sequence of objects of various weights in a random order, and must choose irrevocably to accept or reject each item. There is a further constraint that the set of items selected must…
We consider a distributed learning setting where each agent/learner holds a specific parametric model and data source. The goal is to integrate information across a set of learners to enhance the prediction accuracy of a given learner. A…
Most recent results in matrix completion assume that the matrix under consideration is low-rank or that the columns are in a union of low-rank subspaces. In real-world settings, however, the linear structure underlying these models is…