Related papers: Relative Entropy, Probabilistic Inference and AI
This dissertation investigates relative entropies, also called generalized divergences, and how they can be used to characterize information-theoretic tasks in quantum information theory. The main goal is to further refine characterizations…
Even today, the concept of entropy is perceived by many as quite obscure. The main difficulty is analyzed as being fundamentally due to the subjectivity and anthropocentrism of the concept that prevent us to have a sufficient distance to…
Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…
What is a question? According to Cox a question can be identified with the set of assertions that constitute possible answers. In this paper we propose a different approach that combines the notion that questions are requests for…
The relative entropy of two n-party quantum states is an important quantity exhibiting, for example, the extent to which the two states are different. The relative entropy of the states formed by reducing two n-party to a smaller number $m$…
We establish convergence in the diffusive limit from entropy weak solutions of the equations of compressible gas dynamics with friction to the porous media equation away from vacuum. The result is based on a Lyapunov type of functional…
Let $A$ and $B$ be two accretive operators. We first introduce the weighted geometric mean of $A$ and $B$ together with some related properties. Afterwards, we define the relative entropy as well as the Tsallis entropy of $A$ and $B$. The…
Information theory has been taken as a prospective tool for quantifying the complexity of complex networks. In this paper, we first study the information entropy or uncertainty of a path using the information theory. Then we apply the path…
The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been…
Some general considerations on the notion of entropy in physics are presented. An attempt is made to clarify the question of the differentiation between physical entropy (the Clausius-Boltzmann one) and quantities called entropies…
We obtain formulas for Petz-R\'enyi and Umegaki relative entropy from the idea of distribution of a positive selfadjoint operator. Classical results on R\'enyi and Kullback-Leibler divergences are applied to obtain new results and new…
We prove that the quantum relative entropy is a rate function in large deviation principle. Next, we define information criteria for quantum states and estimate the accuracy of the use of them. Most of the results in this paper are…
Product probability property, known in the literature as statistical independence, is examined first. Then generalized entropies are introduced, all of which give generalizations to Shannon entropy. It is shown that the nature of the…
We ask the question whether entropy accumulates, in the sense that the operationally relevant total uncertainty about an $n$-partite system $A = (A_1, \ldots A_n)$ corresponds to the sum of the entropies of its parts $A_i$. The Asymptotic…
The problem addressed concerns the determination of the average number of successive attempts of guessing a word of a certain length consisting of letters with given probabilities of occurrence. Both first- and second-order approximations…
This thesis develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and…
Many thermodynamic relations involve inequalities, with equality if a process does not involve dissipation. In this article we provide equalities in which the dissipative contribution is shown to involve the relative entropy (a.k.a.…
A criterion and necessary conditions for convergence (local continuity) of the quantum relative entropy are obtained. Some applications of these results are considered. In particular, the preservation of local continuity of the quantum…
For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one…