Related papers: Shannon entropy for imprecise and under-defined or…
Despite the wide usage of information as a concept in science, we have yet to develop a clear & concise scientific definition. This paper is aimed at laying the foundations for a new theory concerning the mechanics of information alongside…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
The fractional order generalization of Shannon entropy proposed by Ubriaco has been studied for discrete distributions. In the current paper, we conduct a detailed study of the continuous analogue of this entropy termed as fractional…
I consider the effect of a finite sample size on the entropy of a sample of independent events. I propose formula for entropy which satisfies Shannon's axioms, and which reduces to Shannon's entropy when sample size is infinite. I discuss…
Topos formalism for quantum mechanics is interpreted in a broader, information retrieval perspective. Contexts, its basic components, are treated as sources of information. Their interplay, called daseinisation, defined in purely logical…
Estimating entropies from limited data series is known to be a non-trivial task. Naive estimations are plagued with both systematic (bias) and statistical errors. Here, we present a new 'balanced estimator' for entropy functionals Shannon,…
It is shown that the standard expression for the information entropy, originally due to Shannon, is only valid for a particular set of states. For the general case of statistical mechanics, one needs to include an additional term in the…
The Shannon information-entropy uncertainty (in brief as "information uncertainty") is used to analyze the fragments in the measured 140$A$ MeV $^{40, 48}$Ca + $^{9}$Be and $^{58, 64}$Ni + $^{9}$Be reactions. A scaling phenomenon is found…
In this chapter, a statistical measure of complexity and the Fisher-Shannon information product are introduced and their properties are discussed. These measures are based on the interplay between the Shannon information, or a function of…
In this paper, we present a theoretical discussion on AI deep learning neural network uncertainty investigation based on the classical Rademacher complexity and Shannon entropy. First it is shown that the classical Rademacher complexity and…
In this work we determine and discuss the entropic uncertainty measures of Shannon type for all the discrete stationary states of the multidimensional harmonic systems directly in terms of the states' hyperquantum numbers, the…
A method of representing probabilistic aspects of quantum systems is introduced by means of a density function on the space of pure quantum states. In particular, a maximum entropy argument allows us to obtain a natural density function…
Transfer entropy is capable of capturing nonlinear source-destination relations between multi-variate time series. It is a measure of association between source data that are transformed into destination data via a set of linear…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
This paper is prepared as a contribution to the proceedings after the 12th ICSSUR/Feynfest Conference held in Foz do Iguacu (Brazil) from 2 to 6 May 2011. In the first part I briefy report the topic of entropic uncertainty relations for…
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…
A message of any sort can be regarded as a source of information. Claude. E. Shannon showed in the last century that information ("what we don't already know") is equivalent to the entropy as defined in statistical mechanics. A string of…
Shannon based his information theory on the notion of probability measures as it we developed by Kolmogorov. In this paper we study some fundamental problems in information theory based on expectation measures. In the theory of expectation…
We propose a new interpretation of measures of information and disorder by connecting these concepts to group theory in a new way. Entropy and group theory are connected here by their common relation to sets of permutations. A combinatorial…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…