Related papers: Synergy, suppression and immorality: forward diffe…
The following problem is considered: given a joint distribution $P_{XY}$ and an event $E$, bound $P_{XY}(E)$ in terms of $P_XP_Y(E)$ (where $P_XP_Y$ is the product of the marginals of $P_{XY}$) and a measure of dependence of $X$ and $Y$.…
We study belief revision when information is represented by a set of probability distributions, or general information. General information extends the standard event notion while including qualitative information (A is more likely than B),…
The description of the dynamics of complex systems, in particular the capture of the interaction structure and causal relationships between elements of the system, is one of the central questions of interdisciplinary research. While the…
We define {\em predictive information} $I_{\rm pred} (T)$ as the mutual information between the past and the future of a time series. Three qualitatively different behaviors are found in the limit of large observation times $T$: $I_{\rm…
How to estimate heterogeneity, e.g. the effect of some variable differing across observations, is a key question in political science. Methods for doing so make simplifying assumptions about the underlying nature of the heterogeneity to…
We address the excess entropy, which is a measure of complexity for stationary time series, from the ordinal point of view. We show that the permutation excess entropy is equal to the mutual information between two adjacent semi-infinite…
This article introduces a model-agnostic approach to study statistical synergy, a form of emergence in which patterns at large scales are not traceable from lower scales. Our framework leverages various multivariate extensions of Shannon's…
Bayes' theorem incorporates distinct types of information through the likelihood and prior. Direct observations of state variables enter the likelihood and modify posterior probabilities through consistent updating. Information in terms of…
Strong subadditivity goes beyond the tensored subsystem and commuting operator models. As previously noted by Petz and later by Araki and Moriya, two subalgebras of observables satisfy a generalized SSA-like inequality if they form a…
High-order phenomena play crucial roles in many systems of interest, but their analysis is often highly nontrivial. There is a rich literature providing a number of alternative information-theoretic quantities capturing high-order…
In the social sciences we are often interested in comparing models specified by parametric equality or inequality constraints. For instance, when examining three group means $\{ \mu_1, \mu_2, \mu_3\}$ through an analysis of variance…
Imitation is a basic updating mechanism for strategy evolution in structured populations, determining how individuals sample social information and translate it into behavioral changes. Higher-order networks, such as hypergraphs, generalize…
The idea of intersectionality has become a frequent topic of discussion both in academic sociology, as well as among popular movements for social justice such as Black Lives Matter, intersectional feminism, and LGBT rights.…
In statistical inference, uncertainty is unknown and all models are wrong. That is to say, a person who makes a statistical model and a prior distribution is simultaneously aware that both are fictional candidates. To study such cases,…
We consider an approximation scheme for multivariate information assuming that synergistic information only appearing in higher order joint distributions is suppressed, which may hold in large classes of systems. Our approximation scheme…
Algorithmic fairness is becoming increasingly important in data mining and machine learning. Among others, a foundational notation is group fairness. The vast majority of the existing works on group fairness, with a few exceptions,…
Weighted Updating generalizes Bayesian updating, allowing for biased beliefs by weighting the likelihood function and prior distribution with positive real exponents. I provide a rigorous foundation for the model by showing that…
The conditional mutual information quantifies the conditional dependence of two random variables. It has numerous applications; it forms, for example, part of the definition of transfer entropy, a common measure of the causal relationship…
New families of Fisher information and entropy power inequalities for sums of independent random variables are presented. These inequalities relate the information in the sum of $n$ independent random variables to the information contained…
Most existing works on disentangled representation learning are solely built upon an marginal independence assumption: all factors in disentangled representations should be statistically independent. This assumption is necessary but…