Related papers: Groups, Information Theory and Einstein's Likeliho…
We generalize the Shannon's information theory in a nonadditive way by focusing on the source coding theorem. The nonadditive information content we adopted is consistent with the concept of the form invariance structure of the nonextensive…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
We consider the Shannon-Khinchin axiomatic systems for the characterization of generalized entropies such as Sharma-Mital and Frank-Daffertshofer entropy. We provide the generalization of Shannon-Khinchin axioms and give the corresponding…
This paper studies properties of entropy functions that are induced by groups and subgroups. We showed that many information theoretic properties of those group induced entropy functions also have corresponding group theoretic…
A method of representing probabilistic aspects of quantum systems is introduced by means of a density function on the space of pure quantum states. In particular, a maximum entropy argument allows us to obtain a natural density function…
Even if a probability distribution is properly normalizable, its associated Shannon (or von Neumann) entropy can easily be infinite. We carefully analyze conditions under which this phenomenon can occur. Roughly speaking, this happens when…
We are now witnessing a rapid growth of a new part of group theory which has become known as "statistical group theory". A typical result in this area would say something like ``a random element (or a tuple of elements) of a group G has a…
We live in the information age. Claude Shannon, as the father of the information age, gave us a theory of communications that quantified an "amount of information," but, as he pointed out, "no concept of information itself was defined."…
It is shown that the standard expression for the information entropy, originally due to Shannon, is only valid for a particular set of states. For the general case of statistical mechanics, one needs to include an additional term in the…
We show how generalized Gibbs-Shannon entropies can provide new insights on the statistical properties of texts. The universal distribution of word frequencies (Zipf's law) implies that the generalized entropies, computed at the word level,…
The fundamental goal of information theory is to characterize complex operational tasks using efficiently computable information quantities, Shannon's capacity formula being the prime example of this. However, many tasks in quantum…
Given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems', information-theoretic analyses became increasingly necessary during the past few decades in the…
When at equilibrium, large-scale systems obey conventional thermodynamics because they belong to microscopic configurations (or states) that are typical. Crucially, the typical states usually represent only a small fraction of the total…
Entropic uncertainty relations are interesting in their own rights as well as for a lot of applications. Keeping this in mind, we try to make the corresponding inequalities as tight as possible. The use of parametrized entropies also allows…
The quantification of the complexity of networks is, today, a fundamental problem in the physics of complex systems. A possible roadmap to solve the problem is via extending key concepts of information theory to networks. In this paper we…
Many complex systems are characterized by non-Boltzmann distribution functions of their statistical variables. If one wants to -- justified or not -- hold on to the maximum entropy principle for complex statistical systems (non-Boltzmann)…
Zipf's law, and power laws in general, have attracted and continue to attract considerable attention in a wide variety of disciplines - from astronomy to demographics to software structure to economics to linguistics to zoology, and even…
We propose an information theoretic model for sociological networks. The model is a microcanonical ensemble of states and particles. The states are the possible pairs of nodes (i.e. people, sites and alike) which exchange information. The…
Given a universe of discourse X-a domain of possible outcomes-an experiment may consist of selecting one of its elements, subject to the operation of chance, or of observing the elements, subject to imprecision. A priori uncertainty about…
Information theory provides shortcuts which allow one to deal with complex systems. The basic idea one uses for this purpose is the maximum entropy principle developed by Jaynes. However, an extension of this maximum entropy principle to…