Related papers: An Information-Theoretic Proof of the Kac--Bernste…
Bell inequalities may only be derived, if hidden variables do not depend on the experimental settings. The stochastic independence of hidden and setting variables is called: freedom of choice, free will, measurement independence or no…
We present a theory of information expressed solely in terms of which transformations of physical systems are possible and which are impossible - i.e. in constructor-theoretic terms. Although it includes conjectured laws of physics that are…
A random dense countable set is characterized (in distribution) by independence and stationarity. Two examples are `Brownian local minima' and `unordered infinite sample'. They are identically distributed; the former ad hoc proof of this…
Optimal transport and Wasserstein distances are flourishing in many scientific fields as a means for comparing and connecting random structures. Here we pioneer the use of an optimal transport distance between L\'{e}vy measures to solve a…
Any measurement is intended to provide information on a system, namely knowledge about its state. However, we learn from quantum theory that it is generally impossible to extract information without disturbing the state of the system or its…
The max-relative entropy and the conditional min-entropy it induces have become central to one-shot information theory. Both may be expressed in terms of a conic program over the positive semidefinite cone. Recently, it was shown that the…
This paper introduces an information theoretic co-training objective for unsupervised learning. We consider the problem of predicting the future. Rather than predict future sensations (image pixels or sound waves) we predict "hypotheses" to…
Mutual information (MI) is a useful information-theoretic measure to quantify the statistical dependence between two random variables: $X$ and $Y$. Often, we are interested in understanding how the dependence between $X$ and $Y$ in one set…
Consider informative selection of a sample from a finite population. Responses are realized as independent and identically distributed (i.i.d.) random variables with a probability density function (p.d.f.) f, referred to as the…
We introduce the coordination principle, which states that perfect coordination, in the form of agreement on a uniformly random output, among N parties is possible only if they share a common cause. This principle is purely causal and can…
If the prior probability distributions of all possible hypothetical true means and all possible observed means of a continuous variable are conditional on the universal set of all numbers (i.e., before the nature of a study is known and a…
Statistical independence is a notion ubiquitous in various fields such as in statistics, probability, number theory and physics. We establish the stability of independence for any pair of random variables by their corresponding Brockwell…
In this paper, we study the linear independence between the distribution of the number of prime factors of integers and that of the largest prime factors of integers. Respectively, under a restriction on the largest prime factors of…
In this paper, we give a Breiman's theorem for conditional dependent random vector, where one component has a regularly-varying-tailed distribution with the index $\alpha\ge0$ and its slowly varying function satisfies a relaxed condition,…
An outlier is a datapoint that is set apart from a sample population. The outlier theorem in algorithmic information theory states that given a computable sampling method, outliers must appear. We present a simple proof to the outlier…
We consider the classical problem of discrete distribution estimation using i.i.d. samples in a novel scenario where additional side information is available on the distribution. In large alphabet datasets such as text corpora, such side…
We propose a general new method, the conditional permutation test, for testing the conditional independence of variables $X$ and $Y$ given a potentially high-dimensional random vector $Z$ that may contain confounding factors. The proposed…
We prove the "Most informative boolean function" conjecture of Courtade and Kumar for high noise $\epsilon \ge 1/2 - \delta$, for some absolute constant $\delta > 0$. Namely, if $X$ is uniformly distributed in $\{0,1\}^n$ and $Y$ is…
Iterative imputation, in which variables are imputed one at a time each given a model predicting from all the others, is a popular technique that can be convenient and flexible, as it replaces a potentially difficult multivariate modeling…
One of the benefits of belief networks and influence diagrams is that so much knowledge is captured in the graphical structure. In particular, statements of conditional irrelevance (or independence) can be verified in time linear in the…