Related papers: A Group Theoretic Model for Information
Information theory, introduced by Shannon, has been extremely successful and influential as a mathematical theory of communication. Shannon's notion of information does not consider the meaning of the messages being communicated but only…
We live in the information age. Claude Shannon, as the father of the information age, gave us a theory of communications that quantified an "amount of information," but, as he pointed out, "no concept of information itself was defined."…
This article serves as a brief introduction to the Shannon information theory. Concepts of information, Shannon entropy and channel capacity are mainly covered. All these concepts are developed in a totally combinatorial flavor. Some issues…
We provide a simple physical interpretation, in the context of the second law of thermodynamics, to the information inequality (a.k.a. the Gibbs' inequality, which is also equivalent to the log-sum inequality), asserting that the relative…
We propose a unified framework for both Shannon-Khinchin and Shore-Johnson axiomatic systems. We do it by rephrasing Shannon-Khinchine axioms in terms of generalized arithmetics of Kolmogorov and Nagumo. We prove that the two axiomatic…
Information inequalities govern the ultimate limitations in information theory and as such play an pivotal role in characterizing what values the entropy of multipartite states can take. Proving an information inequality, however, quickly…
We propose an information theoretic model for sociological networks. The model is a microcanonical ensemble of states and particles. The states are the possible pairs of nodes (i.e. people, sites and alike) which exchange information. The…
In the setting where information cannot be verified, we propose a simple yet powerful information theoretical framework---the Mutual Information Paradigm---for information elicitation mechanisms. Our framework pays every agent a measure of…
Mutual information is one of the essential building blocks of information theory. Yet, it is only finitely defined for distributions with fast decaying tails on a countable joint alphabet of two random elements. The unboundedness of mutual…
Information based thermodynamic logic is revisited. It consists of two parts: Part A applies the modern theory of probability in which an arbitrary convex function \phi is employed as an analytic "device" to express information as…
Markov categories are a novel framework to describe and treat problems in probability and information theory. In this work we combine the categorical formalism with the traditional quantitative notions of entropy, mutual information, and…
How to query a dataset in the way of preserving the privacy of individuals whose data is included in the dataset is an important problem. The information privacy model, a variant of Shannon's information theoretic model to the encryption…
We introduce the category of information structures, whose objects are suitable diagrams of measurable sets that encode the possible outputs of a given family of observables and their mutual relationships of refinement; they serve as…
The notion of entropy is ubiquitous both in natural and social sciences. In the last two decades, a considerable effort has been devoted to the study of new entropic forms, which generalize the standard Boltzmann-Gibbs (BG) entropy and are…
Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this…
Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, $S=-\sum_i p_i \log p_i$. Here we note that in physical systems one of these axioms may be violated. For non-ergodic…
The field of Information Theory is founded on Claude Shannon's seminal ideas relating to entropy. Nevertheless, his well-known avoidance of meaning (Shannon, 1948) still persists to this day, so that Information Theory remains poorly…
Information Theory concepts and methodologies conform the background of how communication systems are studied and understood. They are mainly focused on the source-channel-receiver problem and on the asymptotic limits of accuracy and…
Despite the wide usage of information as a concept in science, we have yet to develop a clear & concise scientific definition. This paper is aimed at laying the foundations for a new theory concerning the mechanics of information alongside…
In this work, we derive information-theoretic properties for a modified Tsallis entropy, hereinafter referred to as q-entropy. We introduce the notions of joint q-entropy, conditional q-entropy, relative q-entropy, conditional mutual…