Related papers: Information Measure as Time Complexity
The mutual information (MI) between two random variables is an important correlation measure in data analysis. The Shannon entropy of a joint probability distribution is the variable part under fixed marginals. We aim to minimize and…
We present a quantum information theory that allows for a consistent description of entanglement. It parallels classical (Shannon) information theory but is based entirely on density matrices (rather than probability distributions) for the…
We investigate the relative information efficiency of financial markets by measuring the entropy of the time series of high frequency data. Our tool to measure efficiency is the Shannon entropy, applied to 2-symbol and 3-symbol…
Shannon entropy was defined for probability distributions and then its using was expanded to measure the uncertainty of knowledge for systems with complete information. In this article, it is proposed to extend the using of Shannon entropy…
Shannon information theory is established based on probability and bits, and the communication technology based on this theory realizes the information age. The original goal of Shannon's information theory is to describe and transmit…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
We propose R\'enyi information generating function and discuss its properties. A connection between the R\'enyi information generating function and the diversity index is proposed for discrete type random variables. The relation between the…
The problem of defining and studying complexity of a time series has interested people for years. In the context of dynamical systems, Grassberger has suggested that a slow approach of the entropy to its extensive asymptotic limit is a sign…
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
There is no single universally accepted definition of "Complexity". There are several perspectives on complexity and what constitutes complex behaviour or complex systems, as opposed to regular, predictable behaviour and simple systems. In…
We investigate how to measure and define the entropy of a simple chaotic system, three hard spheres on a ring. A novel approach is presented, which does not assume the ergodic hypothesis. It consists of transforming the particles collision…
A message of any sort can be regarded as a source of information. Claude. E. Shannon showed in the last century that information ("what we don't already know") is equivalent to the entropy as defined in statistical mechanics. A string of…
Integrated Information Theory (IIT) is a prominent theory of consciousness that has at its centre measures that quantify the extent to which a system generates more information than the sum of its parts. While several candidate measures of…
We establish a precise connection between statistical significance in dependence testing and information-theoretic dependence as quantified by Shannon mutual information (MI). In the absence of prior distributional information, we consider…
Given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems', information-theoretic analyses became increasingly necessary during the past few decades in the…
Distributed systems, such as biological and artificial neural networks, process information via complex interactions engaging multiple subsystems, resulting in high-order patterns with distinct properties across scales. Investigating how…
Kolmogorov argued that the concept of information exists also in problems with no underlying stochastic model (as Shannon's information representation) for instance, the information contained in an algorithm or in the genome. He introduced…
Information entropy is applied to the state of knowledge of reaction amplitudes in pseudoscalar meson photoproduction, and a scheme is developed that quantifies the information content of a measured set of polarization observables. It is…
Prediction of events is the challenge in many different disciplines, from meteorology to finance; the more this task is difficult, the more a system is {\it complex}. Nevertheless, even according to this restricted definition, a general…
The average uncertainty associated with words is an information-theoretic concept at the heart of quantitative and computational linguistics. The entropy has been established as a measure of this average uncertainty - also called average…