Related papers: Information Theory with Kernel Methods
Representations of probability measures in reproducing kernel Hilbert spaces provide a flexible framework for fully nonparametric hypothesis tests of independence, which can capture any type of departure from independence, including…
Entropic measures of complexity are able to quantify the information encoded in complex network structures. Several entropic measures have been proposed in this respect. Here we study the relation between the Shannon entropy and the Von…
Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…
For a quantum state undergoing unitary Schr\"odinger time evolution, the von Neumann entropy is constant. Yet the second law of thermodynamics, and our experience, show that entropy increases with time. Ingarden introduced the quantum…
We present the elements of a new approach to the foundations of quantum theory and probability theory which is based on the algebraic approach to integration, information geometry, and maximum relative entropy methods. It enables us to deal…
We define a new divergence of von Neumann algebras using a variational expression that is similar in nature to Kosaki's formula for the relative entropy. Our divergence satisfies the usual desirable properties, upper bounds the sandwiched…
We extend algorithmic information theory to quantum mechanics, taking a universal semicomputable density matrix (``universal probability'') as a starting point, and define complexity (an operator) as its negative logarithm. A number of…
We discuss information-theoretic concepts on infinite-dimensional quantum systems. In particular, we lift the smooth entropy formalism as introduced by Renner and collaborators for finite-dimensional systems to von Neumann algebras. For the…
Informational dependence between statistical or quantum subsystems can be described with Fisher matrix or Fubini-Study metric obtained from variations of the sample/configuration space coordinates. Using these non-covariant objects as…
A large class of strongly correlated quantum systems can be described in certain large-N limits by quadratic in field actions along with self-consistency equations that determine the two-point functions. We use the replica approach and the…
Uncertainty relations provide constraints on how well the outcomes of incompatible measurements can be predicted, and, as well as being fundamental to our understanding of quantum theory, they have practical applications such as for…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
We introduce a divergence measure between data distributions based on operators in reproducing kernel Hilbert spaces defined by kernels. The empirical estimator of the divergence is computed using the eigenvalues of positive definite Gram…
Integral representations of quantum relative entropy, and of the directional second and higher order derivatives of von Neumann entropy, are established, and used to give simple proofs of fundamental, known data processing inequalities: the…
We define correlational (von Neumann) entropy for an individual quantum state of a system whose time-independent hamiltonian contains random parameters and is treated as a member of a statistical ensemble. This entropy is representation…
By introducing Hilbert space and operators, we show how probabilities, approximations and entropy encoding from signal and image processing allow precise formulas and quantitative estimates. Our main results yield orthogonal bases which…
Categorical compositional distributional model of Coecke et al. (2010) suggests a way to combine grammatical composition of the formal, type logical models with the corpus based, empirical word representations of distributional semantics.…
In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…