Related papers: A Novel Distance Matric: Generalized Relative Entr…

Systematic generalization remains challenging for current language models, which are known to be both sensitive to semantically similar permutations of the input and to struggle with known concepts presented in novel contexts. Although…

In this article we describe the applications of the relative entropy framework. In particular uniqueness of an entropy solution is proven for a scalar conservation law, using the notion of measure-valued entropy solutions. Further we survey…

The phenomenon of entropy concentration provides strong support for the maximum entropy method, MaxEnt, for inferring a probability vector from information in the form of constraints. Here we extend this phenomenon, in a discrete setting,…

For general metric theories of gravity, we compare the approach that describes-derives the field equations of gravity as a thermodynamic identity with the one which looks at them from entropy bounds. The comparison is made through the…

We first show how a new definition of entropy, which is intuitively very simple, as a divergence in cluster-size space, leads to a generalized form that is nonextensive for correlated units, but coincides exactly with the conventional one…

Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…

We introduce a class of information measures based on group entropies, allowing us to describe the information-theoretical properties of complex systems. These entropic measures are nonadditive, and are mathematically deduced from a series…

The M-relative distance, denoted by \rho_M is a generalization of the p-relative distance, which was introduced by Ren-Cang Li. We establish necessary and sufficient conditions under which \rho_M is a metric. In two special cases we derive…

As a part of the construction of an information theory based on general probabilistic theories, we propose and investigate the several distinguishability measures and "entropies" in general probabilistic theories. As their applications,…

An universal approximation technique for analysis of different characteristics of states of composite infinite-dimensional quantum systems is proposed and used to prove general results concerning the properties of correlation and…

Euclidean distance matrices (EDM) are symmetric nonnegative matrices with several interesting properties. In this article, we introduce a wider class of matrices called generalized Euclidean distance matrices (GDMs) that include EDMs. Each…

The category of metric spaces is a subcategory of quasi-metric spaces. In this paper the notion of entropy for the continuous maps of a quasi-metric space is extended via spanning and separated sets. Moreover, two metric spaces that are…

The uniqueness theorem for a two-parameter extended relative entropy is proven. This result extends our previous one, the uniqueness theorem for a one-parameter extended relative entropy, to a two-parameter case. In addition, the properties…

We derive a measurement-independent asymptotic continuity bound on the observational entropy for general POVM measurements, making essential use of its property of bounded concavity. The same insight is used to obtain continuity bounds for…

It is well-understood that the robustness of mechanical and robotic control systems depends critically on minimizing sensitivity to arbitrary application-specific details whenever possible. For example, if a system is defined and performs…

Measure-theoretic slow entropy is a more refined invariant than the classical measure-theoretic entropy to characterize the complexity of dynamical systems with subexponential growth rates of distinguishable orbit types. In this paper we…

We introduce a generalization of relative entropy derived from the Wigner-Yanase-Dyson entropy and give a simple, self-contained proof that it is convex. Moreover, special cases yield the joint convexity of relative entropy, and for the map…

Using the generalized entropies which depend on two parameters we propose a set of quantitative characteristics derived from the Information Geometry based on these entropies. Our aim, at this stage, is modest, as we are first constructing…

Entropy metrics (for example, permutation entropy) are nonlinear measures of irregularity in time series (one-dimensional data). Some of these entropy metrics can be generalised to data on periodic structures such as a grid or lattice…

We propose a new way of defining entropy of a system, which gives a general form which may be nonextensive as Tsallis entropy, but is linearly dependent on component entropies, like Renyi entropy, which is extensive. This entropy has a…