Related papers: The Entropy of Parallel Systems
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
Information plays an important role in our understanding of the physical world. We hence propose an entropic measure of information for any physical theory that admits systems, states and measurements. In the quantum and classical world,…
We study the problem of entropy calibration, which asks whether a language model's entropy over generations matches its log loss on human text. Past work found that models are miscalibrated, with entropy per step increasing as generations…
Under certain conditions, the rate of increase of the statistical entropy of a simple, fully chaotic, conservative system is known to be given by a single number, characteristic of this system, the Kolmogorov-Sinai entropy rate. This…
Entropy is a famous and well established concept in physics and engineering that can be used for explanation of basic fundamentals as well it finds applications in several areas, from quantum physics to astronomy, from network communication…
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
This study explores the role of information entropy in understanding nuclear density distributions, including both stable configurations and non-traditional structures such as neutron halos and $\alpha$-clustering. By quantifying the…
Measures of complexity and entropy have not converged to a single quantitative description of levels of organization of complex systems. The need for such a measure is increasingly necessary in all disciplines studying complex systems. To…
Quantum correlation often refers to correlations exhibited by two or more local subsystems under a suitable measurement. These correlations are beyond the framework of classical statistics and the associated classical probability…
We study entropy-bounded computational geometry, that is, geometric algorithms whose running times depend on a given measure of the input entropy. Specifically, we introduce a measure that we call range-partition entropy, which unifies and…
This chapter concerns "control volume analysis", the standard engineering tool for the analysis of flow systems, and its application to entropy balance calculations. Firstly, the principles of control volume analysis are enunciated and…
In this paper entropy based methods are compared and used to measure structural diversity of an ensemble of 21 classifiers. This measure is mostly applied in ecology, whereby species counts are used as a measure of diversity. The measures…
Large reasoning models have demonstrated remarkable performance on complex reasoning tasks, yet the excessive length of their chain-of-thought outputs remains a major practical bottleneck due to high computation cost and poor deployability.…
Identifying the status of individual network units is critical for understanding the mechanism of convolutional neural networks (CNNs). However, it is still challenging to reliably give a general indication of unit status, especially for…
The von Neumann graph entropy is a measure of graph complexity based on the Laplacian spectrum. It has recently found applications in various learning tasks driven by networked data. However, it is computational demanding and hard to…
We study entanglement entropies between the single-particle states of the hole space and its complement in nuclear systems. Analytical results based on the coupled-cluster method show that entanglement entropies are proportional to the…
The ultimate purpose of the statistical analysis of ordinal patterns is to characterize the distribution of the features they induce. In particular, knowing the joint distribution of the pair Entropy-Statistical Complexity for a large class…
We propose the use of entropy, measured from the spatial and flux distribution of pixels in the residual image, as a potential diagnostic and stopping metric for the CLEAN algorithm. Despite its broad success as the standard deconvolution…
The Shannon entropy is used as a basis for applying different lemmas and conjectures concerning the set of gaps between prime numbers G_p , thus estimating several measures of it. The same procedures are applied to artificially created…
Consider communication over a binary-input memoryless output-symmetric channel with low density parity check (LDPC) codes and maximum a posteriori (MAP) decoding. The replica method of spin glass theory allows to conjecture an analytic…