Related papers: Comment on "Universal and accessible entropy estim…
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…
In the paper, we introduce the maximum entropy estimator based on 2-dimensional empirical distribution of the observation sequence of hidden Markov model , when the sample size is big: in that case computing the maximum likelihood estimator…
Language prediction is constrained by informational entropy intrinsic to language, such that there exists a limit to how accurate any language model can become and equivalently a lower bound to language compression. The most efficient…
Information plays an important role in our understanding of the physical world. We hence propose an entropic measure of information for any physical theory that admits systems, states and measurements. In the quantum and classical world,…
In these notes we will give an overview and road map for a definition and characterization of (relative) entropy for both classical and quantum systems. In other words, we will provide a consistent treatment of entropy which can be applied…
We study the scaling behavior of the entanglement entropy of two dimensional conformal quantum critical systems, i.e. systems with scale invariant wave functions. They include two-dimensional generalized quantum dimer models on bipartite…
Observational entropy captures both the intrinsic uncertainty of a thermodynamic state and the lack of knowledge due to coarse-graining. We demonstrate two interpretations of observational entropy, one as the statistical deficiency…
Information entropy and its extension, which are important generalization of entropy, have been applied in many research domains today. In this paper, a novel generalized relative entropy is constructed to avoid some defects of traditional…
The paper describes an approach to measuring convergence of an algorithm to its result in terms of an entropy-like function of partitions of its inputs of a given length. The goal is to look at the algorithmic data processing from the…
It is well known that loss of information about a system, for some observer, leads to an increase in entropy as perceived by this observer. We use this to propose an alternative approach to decoherence in quantum field theory in which the…
Maximum entropy models are increasingly being used to describe the collective activity of neural populations with measured mean neural activities and pairwise correlations, but the full space of probability distributions consistent with…
An universal approximation technique for analysis of different characteristics of states of composite infinite-dimensional quantum systems is proposed and used to prove general results concerning the properties of correlation and…
In this paper, we investigate and compare two well-developed definitions of entropy relevant for describing the dynamics of isolated quantum systems: bipartite entanglement entropy and observational entropy. In a model system of interacting…
We consider the problem of finite sample corrections for entropy estimation. New estimates of the Shannon entropy are proposed and their systematic error (the bias) is computed analytically. We find that our results cover correction…
We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…
Physics concepts have often been borrowed and independently developed by other fields of science. In this perspective a significant example is that of entropy in Information Theory. The aim of this paper is to provide a short and…
To quantify the complexity of a system, entropy-based methods have received considerable critical attentions in real-world data analysis. Among numerous entropy algorithms, amplitude-based formulas, represented by Sample Entropy, suffer…
Transfer entropy is a widely used measure for quantifying directed information flows in complex systems. While the challenges of estimating transfer entropy for continuous data are well known, it has two major shortcomings for data of…
A scheme is presented to extract detailed dynamical signatures from successive measurements of complex systems. Relative entropy based time series tools are used to quantify the gain in predictive power of increasing past knowledge. By…