Related papers: Stripping syntax from complexity: An information-t…
Does semantic communication require a semantic information theory parallel to Shannon's information theory, or can Shannon's work be generalized for semantic communication? This paper advocates for the latter and introduces a semantic…
Understanding the origins of complexity is a fundamental challenge with implications for biological and technological systems. Network theory emerges as a powerful tool to model complex systems. Networks are an intuitive framework to…
From a traditional point of view, the value of information does not change during transmission. The Shannon information theory considers information transmission as a statistical phenomenon for measuring the communication channel capacity.…
Despite the wide usage of information as a concept in science, we have yet to develop a clear & concise scientific definition. This paper is aimed at laying the foundations for a new theory concerning the mechanics of information alongside…
The Shannon-Weaver model of linear information transmission is extended with two loops potentially generating redundancies: (i) meaning is provided locally to the information from the perspective of hindsight, and (ii) meanings can be…
Semantic communication is regarded as the breakthrough beyond the Shannon paradigm, which transmits only semantic information to significantly improve communication efficiency. This article introduces a framework for generalized semantic…
The constituents of a complex system exchange information to function properly. Their signalling dynamics often leads to the appearance of emergent phenomena, such as phase transitions and collective behaviors. While information exchange…
We present a theory for modeling the structure of information and a language (Quanta) expressing the theory. Unlike Shannon's information theory, which focuses on the amount of information in an information system, we focus on the structure…
The information content of a source is defined in terms of the minimum number of bits needed to store the output of the source in a perfectly recoverable way. A similar definition can be given in the case of quantum sources, with qubits…
A message of any sort can be regarded as a source of information. Claude. E. Shannon showed in the last century that information ("what we don't already know") is equivalent to the entropy as defined in statistical mechanics. A string of…
Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome's plausibility. Information measures based on…
We introduce an asymmetric distance in the space of learning tasks, and a framework to compute their complexity. These concepts are foundational for the practice of transfer learning, whereby a parametric model is pre-trained for a task,…
The historical development has lead to the decay of Natural Philosophy which until 19th century included all of our knowledge about the physical world into the growing multitude of specialized sciences. The focus on the in-depth enquiry…
We compare the elementary theories of Shannon information and Kolmogorov complexity, the extent to which they have a common purpose, and where they are fundamentally different. We discuss and relate the basic notions of both theories:…
Stochastic networks represent very important subject of research because they have been found in almost all branches of modern science, including also sociology and economy. We provide a information theory point of view, mostly based on its…
Science is a constructed narrative of the natural world based on information gathering and its subsequent analysis. In this essay, we develop a novel approach to the epistemic foundations of the scientific narrative, as based on our…
To comprehend the multipartite organization of large-scale biological and social systems, we introduce a new information theoretic approach that reveals community structure in weighted and directed networks. The method decomposes a network…
Kolmogorov argued that the concept of information exists also in problems with no underlying stochastic model (as Shannon's information representation) for instance, the information contained in an algorithm or in the genome. He introduced…
This article serves as a supplement to the recently published call for participation in a Research Topic [1] that is timed to commemorate the 75th anniversary of Shannon's pioneering 1948 paper [2]. Here, we include some citations of key…
We present a theoretical framework that extends classical information theory to finite and structured systems by redefining redundancy as a fundamental property of information organization rather than inefficiency. In this framework,…