Related papers: Information Topology
In this work, we introduce a novel approach based on algebraic topology to enhance graph convolution and attention modules by incorporating local topological properties of the data. To do so, we consider the framework of sheaf neural…
The concept of Shannon entropy of random variables was generalized to measurable functions in general, and to simple functions with finite values in particular. It is shown that the information measure of a function is related to the time…
Statements of Shannon's Noiseless Coding Theorem by various authors, including the original, are reviewed and clarified. Traditional statements of the theorem are often unclear as to when it applies. A new notation is introduced and the…
A framework integrating information theory and network science is proposed, giving rise to a potentially new area. By incorporating and integrating concepts such as complexity, coding, topological projections and network dynamics, the…
This paper presents the computational methods of information cohomology applied to genetic expression in and in the companion paper and proposes its interpretations in terms of statistical physics and machine learning. In order to further…
Knowledge production is often viewed as an endogenous process in which discovery arises through the recombination of existing theories, findings, and concepts. Yet given the vast space of potential recombinations, not all are equally…
An information theory description of finite systems explicitly evolving in time is presented. We impose a MaxEnt variational principle on the Shannon entropy at a given time while the constraints are set at a former time. The resulting…
Bayesian integral functional measure of entropy-uncertainty (EF) on trajectories of Markov multi-dimensional diffusion process is cutting off by interactive impulses (controls). Each cutoff minimax of EF superimposes and entangles…
Information (I) is defined as the amount of the data after data compression. The first law of information theory: the total amount of data L (the sum of entropy S and information I) of an isolated system remains unchanged. The second law of…
We live in the information age. Claude Shannon, as the father of the information age, gave us a theory of communications that quantified an "amount of information," but, as he pointed out, "no concept of information itself was defined."…
Topological techniques have become a popular tool for studying information flows in neural networks. In particular, simplicial homology theory is used to analyze how cognitive representations of space emerge from large conglomerates of…
Researchers have proposed formal definitions of quantitative information flow based on information theoretic notions such as the Shannon entropy, the min entropy, the guessing entropy, and channel capacity. This paper investigates the…
The code base of software projects evolves essentially through inserting and removing information to and from the source code. We can measure this evolution via the elements of information - tokens, words, nodes - of the respective…
Topology identification and inference of processes evolving over graphs arise in timely applications involving brain, transportation, financial, power, as well as social and information networks. This chapter provides an overview of graph…
We provide an axiomatic characterization of information fusion, on the basis of which we define an information fusion network. Our construction is reminiscent of tangle diagrams in low dimensional topology. Information fusion networks come…
What makes two computational systems equivalent? Topos theory answers with classifying toposes: a system's semantic content is encoded in the geometric theory it classifies, and two presentations are equivalent when their classifying…
This article is an overview of the "SP theory of intelligence". The theory aims to simplify and integrate concepts across artificial intelligence, mainstream computing and human perception and cognition, with information compression as a…
There are (at least) three approaches to quantifying information. The first, algorithmic information or Kolmogorov complexity, takes events as strings and, given a universal Turing machine, quantifies the information content of a string as…
Given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems', information-theoretic analyses became increasingly necessary during the past few decades in the…
We survey and introduce concepts and tools located at the intersection of information theory and network biology. We show that Shannon's information entropy, compressibility and algorithmic complexity quantify different local and global…