Related papers: Information Theory based on Non-additive Informati…
Information theory gives rise to a novel method for causal skeleton discovery by expressing associations between variables as tensors. This tensor-based approach reduces the dimensionality of the data needed to test for conditional…
The information spectrum approach gives general formulae for optimal rates of codes in many areas of information theory. In this paper the quantum spectral divergence rates are defined and properties of the rates are derived. The entropic…
In an effort to develop the foundations for a non-stochastic theory of information, the notion of $\delta$-mutual information between uncertain variables is introduced as a generalization of Nair's non-stochastic information functional.…
Information theory plays a central role in establishing fundamental limits on what any learning or estimation algorithm can -- and cannot -- achieve, regardless of computational power. In this chapter, we provide an introduction to these…
We study the relationship between the electronic spectrum structure and the configurational order of one-dimensional quasiperiodic systems. We take the Fibonacci case as an specific example, but the ideas outlined here may be useful to…
A classical random variable can be faithfully compressed into a sequence of bits with its expected length lies within one bit of Shannon entropy. We generalize this variable-length and faithful scenario to the general quantum source…
Document classification is the detection specific content of interest in text documents. In contrast to the data-driven machine learning classifiers, knowledge-based classifiers can be constructed based on domain specific knowledge, which…
After Shannon, entropy becomes a fundamental quantity to describe not only uncertainity or chaos of a system but also information carried by the system. Shannon's important discovery is to give a mathematical expression of the mutual…
This article proposes a new two-parameter generalized entropy, which can be reduced to the Tsallis and the Shannon entropy for specific values of its parameters. We develop a number of information-theoretic properties of this generalized…
This paper proposes a novel entropy encoding technique for lossless data compression. Representing a message string by its lexicographic index in the permutations of its symbols results in a compressed version matching Shannon entropy of…
A probabilistic source is defined as the set of infinite words (over a given denumerable alphabet) endowed with a probability $\mu$. The paper deals with general binary sources where the distribution of any symbol (0 or 1) may depend on an…
We introduce algorithmic information theory, also known as the theory of Kolmogorov complexity. We explain the main concepts of this quantitative approach to defining `information'. We discuss the extent to which Kolmogorov's and Shannon's…
While information theory has been introduced to characterize the fundamental limitations of control and filtering for a few decades, the existing information-theoretic methods are indirect and cumbersome for analyzing the limitations of…
Over the past decade quantum information theory has developed into a vigorous field of research despite the fact that quantum information, as a precise concept, is undefined. Indeed the very idea of viewing quantum states as carriers of…
The interactions between three or more random variables are often nontrivial, poorly understood, and yet, are paramount for future advances in fields such as network information theory, neuroscience, genetics and many others. In this work,…
Does information play a significant role in the foundations of physics? Information is the abstraction that allows us to refer to the states of systems when we choose to ignore the systems themselves. This is only possible in very…
We propose R\'enyi information generating function and discuss its properties. A connection between the R\'enyi information generating function and the diversity index is proposed for discrete type random variables. The relation between the…
We demonstrate that Shannon's information entropy and the thermodynamic entropy of Boltzmann and Gibbs are quantitatively equivalent for real condensed-matter systems. By interpreting atomic configurations as information sources, we compute…
In this paper we extend our recent results [Physica A340 (2004)110] on q-nonextensive statistics with non-Tsallis entropies. In particular, we combine an axiomatics of Renyi with the q-deformed version of Khinchin axioms to obtain the…
Uncertainty relations are central to quantum physics. While they were originally formulated in terms of variances, they have later been successfully expressed with entropies following the advent of Shannon information theory. Here, we…