Related papers: A Logarithmic Decomposition and a Signed Measure S…
Token identity is semantic information for measurement-bearing expressions. Intervals, dimension tags, and token-erased syntax can say what values a measured leaf may take, but they cannot say whether two occurrences name the same…
The paper describes an approach to measuring convergence of an algorithm to its result in terms of an entropy-like function of partitions of its inputs of a given length. The goal is to look at the algorithmic data processing from the…
A consistent generalization of statistical mechanics is obtained by applying the maximum entropy principle to a trace-form entropy and by requiring that physically motivated mathematical properties are preserved. The emerging…
We introduce a notion of a point-wise entropy of measures (i.e local entropy) called neutralized local entropy, and compare it with the Brin-Katok local entropy. We show that the neutralized local entropy coincides with Brin-Katok local…
Ranked set sampling is a sampling design which has a wide range of applications in industrial statistics, and environmental and ecological studies, etc.. It is well known that ranked set samples provide more Fisher information than simple…
The rapid scaling of artificial intelligence models has revealed a fundamental tension between model capacity (storage) and inference efficiency (computation). While classical information theory focuses on transmission and storage limits,…
The concept of Shannon entropy of random variables was generalized to measurable functions in general, and to simple functions with finite values in particular. It is shown that the information measure of a function is related to the time…
The formalism of statistical mechanics can be generalized by starting from more general measures of information than the Shannon entropy and maximizing those subject to suitable constraints. We discuss some of the most important examples of…
We investigate the concept of entropy in probabilistic theories more general than quantum mechanics, with particular reference to the notion of information causality recently proposed by Pawlowski et. al. (arXiv:0905.2992). We consider two…
Information flow analysis is a powerful technique for reasoning about the sensitive information exposed by a program during its execution. While past work has proposed information theoretic metrics (e.g., Shannon entropy, min-entropy,…
One requirement of maintaining digital information is storage. With the latest advances in the digital world, new emerging media types have required even more storage space to be kept than before. In fact, in many cases it is required to…
Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable…
Motivation: Entropy measurements on hierarchical structures have been used in methods for information retrieval and natural language modeling. Here we explore its application to semantic similarity. By finding shared ontology terms,…
We investigate certain optimization problems for Shannon information measures, namely, minimization of joint and conditional entropies $H(X,Y)$, $H(X|Y)$, $H(Y|X)$, and maximization of mutual information $I(X;Y)$, over convex regions. When…
The application of the Shannon entropy to study the relationship between information and structures has yielded insights into molecular and material systems. However, the difficulty in directly observing and manipulating atoms and molecules…
In this paper, I expand Shannon's definition of entropy into a new form of entropy that allows integration of information from different random events. Shannon's notion of entropy is a special case of my more general definition of entropy.…
We consider the entropy of sums of independent discrete random variables, in analogy with Shannon's Entropy Power Inequality, where equality holds for normals. In our case, infinite divisibility suggests that equality should hold for…
Information has an entropic character which can be analyzed within the Statistical Theory in molecular systems. R. Landauer and C.H. Bennett showed that a logical copy can be carried out in the limit of no dissipation if the computation is…
We investigate entanglement phase transitions from volume-law to area-law entanglement in a quantum many-body state under continuous position measurement on the basis of the quantum trajectory approach. We find the signatures of the…
The paper makes the observation that all orders of information entropy are equal in signals composed of repeating units of distinct symbols where the units can be classified as a member of a symmetry group. This leads to an improved metric…