相关论文: Axiomatic Origins of Mathematical Entropy: Grading…

Probability theory is fundamental for modeling uncertainty, with traditional probabilities being real and non-negative. Complex probability extends this concept by allowing complex-valued probabilities, opening new avenues for analysis in…

We live in the information age. Claude Shannon, as the father of the information age, gave us a theory of communications that quantified an "amount of information," but, as he pointed out, "no concept of information itself was defined."…

We propose a new interpretation of measures of information and disorder by connecting these concepts to group theory in a new way. Entropy and group theory are connected here by their common relation to sets of permutations. A combinatorial…

A mathematical interpretation of the usual definition of entropy (for a discrete probability distribution or a trace 1 positive operator) is given. This formulation makes some properties of entropy immediate.

Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building…

We conclude a sequence of work by giving near-optimal sketching and streaming algorithms for estimating Shannon entropy in the most general streaming model, with arbitrary insertions and deletions. This improves on prior results that obtain…

This paper has two main objectives. The first is to introduce a notion of entropy that is well suited for the analysis of nonlinear input-output systems that have a Chen-Fliess series representation. The latter is defined in terms of its…

Consider a state of a system with several subsystems. The entropies of the reduced state on different subsystems obey certain inequalities, provided there is an equivalence relation, and a function measuring volumes or weights of…

Entropy is a quantity which is of great importance in physics and chemistry. The concept comes out of thermodynamics, proposed by Rudolf Clausius in his analysis of Carnot cycle and linked by Ludwig Boltzmann to the number of specific ways…

Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…

An observer increases in relative entropy as it receives information from what it is observing. In a system of only an observer and the observed, an increase in the relative entropy of the observer is a decrease in the relative entropy of…

In estimating the complexity of objects, in particular of graphs, it is common practice to rely on graph- and information-theoretic measures. Here, using integer sequences with properties such as Borel normality, we explain how these…

Loosely speaking, the Shannon entropy rate is used to gauge a stochastic process' intrinsic randomness; the statistical complexity gives the cost of predicting the process. We calculate, for the first time, the entropy rate and statistical…

The concept of composability states that entropy of the total system composed of independent subsystems is a function of entropies of the subsystems. Here, the most general pseudoadditivity rule for composable entropy is derived based only…

Information plays an important role in our understanding of the physical world. We hence propose an entropic measure of information for any physical theory that admits systems, states and measurements. In the quantum and classical world,…

The metaphor of a potential epigenetic differentiation landscape broadly suggests that during differentiation a stem cell follows the steepest descending gradient toward a stable equilibrium state which represents the final cell type. It…

Various properties of relative entropy have led to its widespread use in information theory. These properties suggest that relative entropy has a role to play in systems that attempt to perform inference in terms of probability…

We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint…

The present work is dedicated to searching parameters, alternative to entropy, applicable for description of highly organized systems. The general concept has been offered, in which the system complexity and order are functions of the order…

Classification is a machine learning method used in many practical applications: text mining, handwritten character recognition, face recognition, pattern classification, scene labeling, computer vision, natural langage processing. A…