Related papers: GAIT: A Geometric Approach to Information Theory
We investigate fundamental connections between thermodynamics and quantum information theory. First, we show that the operational framework of thermal operations is nonequivalent to the framework of Gibbs-preserving maps, and we comment on…
We present two novel models of document coherence and their application to information retrieval (IR). Both models approximate document coherence using discourse entities, e.g. the subject or object of a sentence. Our first model views text…
This paper introduces the basic concepts of information theory. Based on these concepts, we regard the states in the state space and the types of ideal gases as the symbols in a symbol set to calculate the mixing entropy of ideal gas…
Information theory, i.e. the mathematical analysis of information and of its processing, has become a tenet of modern science; yet, its use in real-world studies is usually hindered by its computational complexity, the lack of coherent…
Information theory has been taken as a prospective tool for quantifying the complexity of complex networks. In this paper, we first study the information entropy or uncertainty of a path using the information theory. Then we apply the path…
What is information? Is it physical? We argue that in a Bayesian theory the notion of information must be defined in terms of its effects on the beliefs of rational agents. Information is whatever constrains rational beliefs and therefore…
The notion of group entropy is proposed. It enables to unify and generalize many different definitions of entropy known in the literature, as those of Boltzmann-Gibbs, Tsallis, Abe and Kaniadakis. Other new entropic functionals are…
Entropy measures of probability distributions are widely used measures in ecology, biology, genetics, and in other fields, to quantify species diversity of a community. Unfortunately, entropy-based diversity indices, or diversity indices…
A new conceptual foundation for the notion of "information" is proposed, based on the concept of a "distinction graph": a graph in which two nodes are connected iff they cannot be distinguished by a particular observer. The "graphtropy" of…
Measuring the distance between ontological elements is fundamental for ontology matching. String-based distance metrics are notorious for shallow syntactic matching. In this exploratory study, we investigate Wasserstein distance targeting…
We propose a theory for coupling matter fields with discrete geometry on higher-order networks, i.e. cell complexes. The key idea of the approach is to associate to a higher-order network the quantum entropy of its metric. Specifically we…
We formulate a geometric framework in which physical laws emerge from restricted access to microscopic information. Measurement constraints are modeled as a gauge symmetry acting on density operators, inducing a gauge-reduced space of…
Causal inference is perhaps one of the most fundamental concepts in science, beginning originally from the works of some of the ancient philosophers, through today, but also weaved strongly in current work from statisticians, machine…
This paper presents case-study results on the application of information theoretic approach to gate-level evolutionary circuit design. We introduce information measures to provide better estimates of synthesis criteria of digital circuits.…
We recall some of the history of the information-theoretic approach to deriving core results in probability theory and indicate parts of the recent resurgence of interest in this area with current progress along several interesting…
Based on the normal distribution and its properties, i.e., average and variance, Fisher works have provided a conceptual framework to identify genotype-phenotype associations. While Fisher intuition has proved fruitful over the past…
The problem of assigning probability distributions which objectively reflect the prior information available about experiments is one of the major stumbling blocks in the use of Bayesian methods of data analysis. In this paper the method of…
Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…
The various entropy bounds that exist in the literature suggest that spacetime is fundamentally discrete, and hint at an underlying relationship between geometry and "information". The foundation of this relationship is yet to be uncovered,…
The rapid scaling of artificial intelligence models has revealed a fundamental tension between model capacity (storage) and inference efficiency (computation). While classical information theory focuses on transmission and storage limits,…