Related papers: Information Measures for Entropy and Symmetry
In statistical physics, useful notions of entropy are defined with respect to some coarse graining procedure over a microscopic model. Here we consider some special problems that arise when the microscopic model is taken to be relativistic…
According to E.T. Jaynes and E.P. Wigner, entropy is an anthropomorphic concept in the sense that in a physical system correspond many thermodynamic systems. The physical system can be examined from many points of view each time examining…
Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability…
Complex systems are found in most branches of science. It is still argued how to best quantify their complexity and to what end. One prominent measure of complexity (the statistical complexity) has an operational meaning in terms of the…
The routine definitions of both entropy, and differential entropy show inconsistencies that make them not reciprocally coherent. We propose a few possible modifications of these quantities so that 1) they no longer show incongruities, 2)…
What is information? Is it physical? We argue that in a Bayesian theory the notion of information must be defined in terms of its effects on the beliefs of rational agents. Information is whatever constrains rational beliefs and therefore…
In classical and quantum information theory, operational quantities such as the amount of randomness that can be extracted from a given source or the amount of space needed to store given data are normally characterized by one of two…
There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the…
We propose a compression-based version of the empirical entropy of a finite string over a finite alphabet. Whereas previously one considers the naked entropy of (possibly higher order) Markov processes, we consider the sum of the…
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…
In its continuous version, the entropy functional measuring the information content of a given probability density may be plagued by a "measure" problem that results from improper weighting of phase space. This issue is addressed…
Given an arbitrary measurement over a system of interest, the outcome of a posterior measurement can be used for improving the statistical estimation of the system state after the former measurement. Here, we realize an…
The notion of software entropy is often invoked to describe the tendency of software systems to become increasingly disordered as they evolve, yet existing approaches to quantify it are largely heuristic. In this work we introduce a formal…
Even today, the concept of entropy is perceived by many as quite obscure. The main difficulty is analyzed as being fundamentally due to the subjectivity and anthropocentrism of the concept that prevent us to have a sufficient distance to…
We propose utilizing entropy as a diagnostic tool to distinguish between constant and dynamical dark energy models. Entropy, a measure of the system's disorder or information content, captures the complexity and evolution of the universe.…
We investigate certain optimization problems for Shannon information measures, namely, minimization of joint and conditional entropies $H(X,Y)$, $H(X|Y)$, $H(Y|X)$, and maximization of mutual information $I(X;Y)$, over convex regions. When…
The principle of entropy increase is not only the basis of statistical mechanics, but also closely related to the irreversibility of time, the origin of life, chaos and turbulence. In this paper, we first discuss the dynamic system…
Entropy is one of the key thermodynamic variables reflecting changes in the state of matter. Unlike other thermodynamic variables, it is well-defined also for nonequilibrium steady states through its relation to information. Applying this…
Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…
An analysis of quantum measurement is presented that relies on an information-theoretic description of quantum entanglement. In a consistent quantum information theory of entanglement, entropies (uncertainties) conditional on measurement…