Related papers: Entropy in Science of Science

During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…

Although compartmental dynamical systems are used in many different areas of science, model selection based on the maximum entropy principle (MaxEnt) is challenging because of the lack of methods for quantifying the entropy for this type of…

Thermodynamic uncertainty relations reveal a fundamental trade-off between the precision of a trajectory observable and entropy production, where the uncertainty of the observable is quantified by its variance. In information theory,…

Entropy measures of probability distributions are widely used measures in ecology, biology, genetics, and in other fields, to quantify species diversity of a community. Unfortunately, entropy-based diversity indices, or diversity indices…

This paper explores the concept of "entropy of knowledge" (EoN) as a framework for understanding the challenges and complexities of scientific discovery. Drawing from principles in thermodynamics and information theory, I propose that the…

This study explores the citation diversity in scholarly literature, analyzing different patterns of citations observed within different countries and academic disciplines. We examine citation distributions across top institutions within…

We show how the Shannon entropy function can be used as a basis to set up complexity measures weighting the economic efficiency of countries and the specialization of products beyond bare diversification. This entropy function guarantees…

There are three ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a…

Entropy can signify different things: For instance, heat transfer in thermodynamics or a measure of information in data analysis. Many entropies have been introduced and it can be difficult to ascertain their different importance and…

Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic…

Organizations generate vast amounts of interconnected content across various platforms. While language models enable sophisticated reasoning for use in business applications, retrieving and contextualizing information from organizational…

Entropic uncertainty relations are interesting in their own rights as well as for a lot of applications. Keeping this in mind, we try to make the corresponding inequalities as tight as possible. The use of parametrized entropies also allows…

Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability…

We propose utilizing entropy as a diagnostic tool to distinguish between constant and dynamical dark energy models. Entropy, a measure of the system's disorder or information content, captures the complexity and evolution of the universe.…

Understanding how network function constrains neural connectivity is a central challenge in neuroscience. An influential approach is to train neural networks with gradient descent on cognitive tasks and characterize the resulting…

Recently, the Shannon entropy, which was introduced originally as a measure of information amount, has been widely used as a useful index of various diversities such as biodiversity and geodiversity. In this work we have evaluated the…

The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…

Ever since Claude Shannon used entropy for his "Mathematical Theory of Communication", entropy has become a buzzword in research circles with scientists applying entropy to describe any phenomena that are reminiscent of disorder. In this…

In many complex systems, whether biological or artificial, the thermodynamic costs of communication among their components are large. These systems also tend to split information transmitted between any two components across multiple…

Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building…