Related papers: Information density, structure and entropy in equi…
The logical basis for information theory is the newly developed logic of partitions that is dual to the usual Boolean logic of subsets. The key concept is a "distinction" of a partition, an ordered pair of elements in distinct blocks of the…
Complex systems are characterised by a tight, nontrivial interplay of their constituents, which gives rise to a multi-scale spectrum of emergent properties. In this scenario, it is practically and conceptually difficult to identify those…
Based on the q-exponential distribution which has been observed in more and more physical systems, the varentropy method is used to derive the uncertainty measure of such an abnormal distribution function. The uncertainty measure obtained…
The concept of entropy connects the number of possible configurations with the number of variables in large stochastic systems. Independent or weakly interacting variables render the number of configurations scale exponentially with the…
The information entropies in coordinate and momentum spaces and their sum ($S_r$, $S_k$, $S$) are evaluated for many nuclei using "experimental" densities or/and momentum distributions. The results are compared with the harmonic oscillator…
Physics concepts have often been borrowed and independently developed by other fields of science. In this perspective a significant example is that of entropy in Information Theory. The aim of this paper is to provide a short and…
Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome's plausibility. Information measures based on…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
Information theory allows us to investigate information processing in neural systems in terms of information transfer, storage and modification. Especially the measure of information transfer, transfer entropy, has seen a dramatic surge of…
In this work the information loss in deterministic, memoryless systems is investigated by evaluating the conditional entropy of the input random variable given the output random variable. It is shown that for a large class of systems the…
We compute the low-temperature configurational entropy of a two-dimensional supercooled liquid. Our method, based on a higher-dimensional version of the Grassberger--Procaccia algorithm, can be implemented in a manner that is entirely…
Entropy production is a universal measure of irreversibility and energy dissipation in physical, chemical, and biological systems operating far from equilibrium. However, quantifying and spatiotemporally localising it in complex processes…
Typically, the entropy of an isolated system in equilibrium is calculated by counting the number of accessible microstates, or in more general cases by using the Gibbs formula. In irreversible processes entropy spontaneously increases and…
The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…
This paper provides a perspective on applying the concepts of information thermodynamics, developed recently in non-equilibrium statistical physics, to problems in theoretical neuroscience. Historically, information and energy in…
For a closed-loop control system with a digital channel between the sensor and the controller, the notion of invariance entropy quantifies the smallest average rate of information above which a given compact subset of the state space can be…
This chapter reviews measures of emergence, self-organization, complexity, homeostasis, and autopoiesis based on information theory. These measures are derived from proposed axioms and tested in two case studies: random Boolean networks and…
The problem of estimating entropy production from incomplete information in stochastic thermodynamics is essential for theory and experiments. Whereas a considerable amount of work has been done on this topic, arguably, most of it is…
Systems driven away from thermal equilibrium constantly deliver entropy to their environment. Determining this entropy production requires detailed information about the system's internal states and dynamics. However, in most practical…
The code base of software projects evolves essentially through inserting and removing information to and from the source code. We can measure this evolution via the elements of information - tokens, words, nodes - of the respective…