Related papers: Foundations of Information Theory
Informational entropy is often identified as physical entropy. This is surprising because the two quantities are differently defined and furthermore the former is a subjective quantity while the latter is an objective one. We describe the…
Quantum logical axiomatic systems for quantum theory usually include a postulate that a lattice under consideration is orthomodular. We propose a derivation of orthomodularity from an information-theoretic axiom. This provides conceptual…
The mind-body problem is reviewed in the context of a non-technical account of quantum information theory. The importance of clearly defining: `what is physical?' is highlighted, since only then can we give meaning to the concept…
We present a theory for modeling the structure of information and a language (Quanta) expressing the theory. Unlike Shannon's information theory, which focuses on the amount of information in an information system, we focus on the structure…
Mutual information is one of the essential building blocks of information theory. Yet, it is only finitely defined for distributions with fast decaying tails on a countable joint alphabet of two random elements. The unboundedness of mutual…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
Information theory gives rise to a novel method for causal skeleton discovery by expressing associations between variables as tensors. This tensor-based approach reduces the dimensionality of the data needed to test for conditional…
As a result of the capabilities of quantum information, the science of quantum information processing is now a prospering, interdisciplinary field focused on better understanding the possibilities and limitations of the underlying theory,…
In this note we highlight some connections of UMAP to the basic principles of Information Geometry. Originally, UMAP was derived from Category Theory observations. However, we posit that it also has a natural geometric interpretation.
The theory of distributed conceptual structures, as outlined in this paper, is concerned with the distribution and conception of knowledge. It rests upon two related theories, Information Flow and Formal Concept Analysis, which it seeks to…
Entropy can signify different things: For instance, heat transfer in thermodynamics or a measure of information in data analysis. Many entropies have been introduced and it can be difficult to ascertain their different importance and…
The theoretical base for consciousness, in particular an explanation of how consciousness is defined by the brain, has long been sought by science. We propose a partial theory of consciousness as relations defined by typical data. The…
This paper addresses theoretically correct vs. incorrect ways to apply information theory to point processes.
At this point in time, two major areas of physics, statistical mechanics and quantum mechanics, rest on the foundations of probability and entropy. The last century saw several significant fundamental advances in our understanding of the…
Shannon based his information theory on the notion of probability measures as it we developed by Kolmogorov. In this paper we study some fundamental problems in information theory based on expectation measures. In the theory of expectation…
There are different meanings of foundation of mathematics: philosophical, logical, and mathematical. Here foundations are considered as a theory that provides means (concepts, structures, methods etc.) for the development of whole…
We live in the Information Age, and information has become a critically important component of our life. The success of the Internet made huge amounts of it easily available and accessible to everyone. To keep the flow of this information…
In our previous articles ("The Information and the Matter", v1, v5; more systematically the informational conception is presented in arXiv paper "The Information as Absolute", 2010) it was rigorously shown that the Matter is some…
This paper describes the design principles of methodology of knowledge-oriented information systems based on ontological approach. Such systems implement technology subject-oriented extraction of knowledge from the set of natural language…
Research on summarization has mainly been driven by empirical approaches, crafting systems to perform well on standard datasets with the notion of information Importance remaining latent. We argue that establishing theoretical models of…