Related papers: Redundancy as a Structural Information Principle f…
Existing information-theoretic frameworks based on maximum entropy network ensembles are not able to explain the emergence of heterogeneity in complex networks. Here, we fill this gap of knowledge by developing a classical framework for…
Redundancy needs more precise characterization as it is a major factor in the evolution and robustness of networks of multivariate interactions. We investigate the complexity of such interactions by inferring a connection transitivity that…
The question of how irreversibility can emerge as a generic phenomena when the underlying mechanical theory is reversible has been a long-standing fundamental problem for both classical and quantum mechanics. We describe a mechanism for the…
Redundancy matrices provide insights into the load carrying behavior of statically indeterminate structures. This information can be employed for the design and analysis of structures with regard to certain objectives, for example…
Two information structures are said to be close if, with high probability, there is approximate common knowledge that interim beliefs are close under the two information structures. We define an "almost common knowledge topology" reflecting…
Information theoretic geometry near critical points in classical and quantum systems is well understood for exactly solvable systems. Here we show that renormalization group flow equations can be used to construct the information metric and…
The conventional approach to the general Partial Information Decomposition (PID) problem has been redundancy-based: specifying a measure of redundant information between collections of source variables induces a PID via Moebius-Inversion…
The presence of symmetries imposes a stringent set of constraints on a system. This constrained structure allows intelligent agents interacting with such a system to drastically improve the efficiency of learning and generalization, through…
It is well-known that biological and social interaction networks have a varying degree of redundancy, though a consensus of the precise cause of this is so far lacking. In this paper, we introduce a topological redundancy measure for…
Message passing neural networks iteratively generate node embeddings by aggregating information from neighboring nodes. With increasing depth, information from more distant nodes is included. However, node embeddings may be unable to…
Referring is one of the most basic and prevalent uses of language. How do speakers choose from the wealth of referring expressions at their disposal? Rational theories of language use have come under attack for decades for not being able to…
We take a closer look at the structure of bivariate dependency induced by a pair of predictor random variables $(X_1, X_2)$ trying to synergistically, redundantly or uniquely encode a target random variable $Y$. We evaluate a recently…
We address the problem of distributed state estimation of a linear dynamical process in an attack-prone environment. Recent attempts to solve this problem impose stringent redundancy requirements on the measurement and communication…
Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…
This paper proposes a theoretical framework which models the information provided by retrieval systems in terms of Information Theory. The proposed framework allows to formalize: (i) system effectiveness as an information theoretic…
General characterization of physical measurements is discussed within the framework of a classical information theory. Uncertainty relation for simultaneous measurements of two physical observables is defined in this framework for…
We analyze the performance of redundancy in a multi-type job and multi-type server system. We assume the job dispatcher is unaware of the servers' capacities, and we set out to study under which circumstances redundancy improves the…
We explore the rational, formal and non-formal criteria of consistency, non-triviality and redundancy in the mathematical research now a days. We develop a paradigmatic discussion by analysing the different conceptions of those criteria,…
We introduce an information-theoretic framework that views learning as universal prediction under log loss, characterized through regret bounds. Central to the framework is an effective notion of architecture-based model complexity, defined…
Inclusion of redundant fundamental quantities in SI system has resulted in lot of ambiguities and confusion in modern theories. The incompatibilities between the existing theories can possibly be due to incorrect assumption of fundamental…