Related papers: Studies of entropy measures concerning the gaps of…

Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…

Entropy is critically examined as a fundamental concept in contemporary science and informatics. Although the typical Shannon entropy provides a proper framework for describing the canonical ensemble, it fails to represent adequately the…

Reliable data-driven estimation of Shannon entropy from small data sets, where the number of examples is potentially smaller than the number of possible outcomes, is a critical matter in several applications. In this paper, we introduce a…

We study the gaps between consecutive prime numbers directly through Eratosthenes sieve. Using elementary methods, we identify a recursive relation for these gaps and for specific sequences of consecutive gaps, known as constellations.…

We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…

Entropy has been a common index to quantify the complexity of time series in a variety of fields. Here, we introduce increment entropy to measure the complexity of time series in which each increment is mapped into a word of two letters,…

A number of inequalities for the weighted entropies is proposed, mirroring properties of a standard (Shannon) entropy and related quantities.

We introduce a novel entropy-related function, \textit{non-repeatability}, designed to capture dynamical behaviors in complex systems. Its normalized form, \textit{mutability}, has been previously applied in statistical physics as a…

Entropic measures of complexity are able to quantify the information encoded in complex network structures. Several entropic measures have been proposed in this respect. Here we study the relation between the Shannon entropy and the Von…

We explore the relation between entanglement entropy of quantum many body systems and the distribution of corresponding, properly selected, observables. Such a relation is necessary to actually measure the entanglement entropy. We show that…

Entropies must correspond to mean values for them to be measurable. The Shannon entropy corresponds to the weighted arithmetic mean, whereas the Renyi entropy corresponds to the exponential mean. These means refer to code lengths, which are…

This short book is an elementary course on entropy, leading up to a calculation of the entropy of hydrogen gas at standard temperature and pressure. Topics covered include information, Shannon entropy and Gibbs entropy, the principle of…

After Shannon, entropy becomes a fundamental quantity to describe not only uncertainity or chaos of a system but also information carried by the system. Shannon's important discovery is to give a mathematical expression of the mutual…

The quality of image encryption is commonly measured by the Shannon entropy over the ciphertext image. However, this measurement does not consider to the randomness of local image blocks and is inappropriate for scrambling based image…

We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…

Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…

It is not obvious how to extend Shannon's original information entropy to higher dimensions, and many different approaches have been tried. We replace the English text symbol sequence originally used to illustrate the theory by a discrete,…

Thermodynamic entropy is determined by a heat measurement through the Clausius equality. The entropy then formalizes a fundamental limitation of operations by the second law of thermodynamics. The entropy is also expressed as the Shannon…

Shannon entropy was defined for probability distributions and then its using was expanded to measure the uncertainty of knowledge for systems with complete information. In this article, it is proposed to extend the using of Shannon entropy…

We consider an isomorphism invariant for measure-preserving systems - types of generalized entropy convergence rates. We show the connections of this invariant with the types of Shannon entropy convergence rates. In the case when they…