Related papers: Information Theory with Kernel Methods
In many problems in data mining and machine learning, data items that need to be clustered or classified are not points in a high-dimensional space, but are distributions (points on a high dimensional simplex). For distributions, natural…
From the output produced by a memoryless deletion channel with a uniformly random input of known length $n$, one obtains a posterior distribution on the channel input. The difference between the Shannon entropy of this distribution and that…
This dissertation investigates relative entropies, also called generalized divergences, and how they can be used to characterize information-theoretic tasks in quantum information theory. The main goal is to further refine characterizations…
Quantum mechanics and information theory are among the most important scientific discoveries of the last century. Although these two areas initially developed separately it has emerged that they are in fact intimately related. In this…
A number of recent works have sought to generalize the Kolmogorov-Sinai entropy of probability-preserving transformations to the setting of Markov operators acting on the integrable functions on a probability space $(X,\mu)$. These have…
In information theory, one major goal is to find useful functions that summarize the amount of information contained in the interaction of several random variables. Specifically, one can ask how the classical Shannon entropy, mutual…
Shannon entropy and Fisher information functionals are known to quantify certain information-theoretic properties of continuous probability distributions of various origins. We carry out a systematic study of these functionals, while…
The role of kernels is central to machine learning. Motivated by the importance of power-law distributions in statistical modeling, in this paper, we propose the notion of power-law kernels to investigate power-laws in learning problem. We…
We prove that kernel covariance embeddings lead to information-theoretically perfect separation of distinct continuous probability distributions. In statistical terms, we establish that testing for the \emph{equality} of two non-atomic…
Given the algebra of observables of a quantum system subject to selection rules, a state can be represented by different density matrices. As a result, different von Neumann entropies can be associated with the same state. Motivated by a…
We consider the problem of learning a set from random samples. We show how relevant geometric and topological properties of a set can be studied analytically using concepts from the theory of reproducing kernel Hilbert spaces. A new kind of…
Relations between Shannon entropy and Renyi entropies of integer order are discussed. For any N-point discrete probability distribution for which the Renyi entropies of order two and three are known, we provide an lower and an upper bound…
Experimental designs are tools which can dramatically reduce the number of simulations required by time-consuming computer codes. Because we don't know the true relation between the response and inputs, designs should allow one to fit a…
In this paper, I expand Shannon's definition of entropy into a new form of entropy that allows integration of information from different random events. Shannon's notion of entropy is a special case of my more general definition of entropy.…
This paper introduces a novel approach to probabilistic deep learning, kernel density matrices, which provide a simpler yet effective mechanism for representing joint probability distributions of both continuous and discrete random…
The theory of positive kernels and associated reproducing kernel Hilbert spaces, especially in the setting of holomorphic functions, has been an important tool for the last several decades in a number of areas of complex analysis and…
We use phase space distributions specifically, the Wigner distribution (WD) and Husimi distribution (HD) to investigate certain information-theoretic measures as descriptors for a given system. We extensively investigate and analyze…
An inversion method is formulated for extracting entanglement-related information on two-particle interactions in a one-dimensional system from measurable one-particle position- and momentum-distribution functions. The method is based on a…
An entropic approach to formulating uncertainty relations for the number-annihilation pair is considered. We construct some normal operator that traces the annihilation operator as well as commuting quadratures with a complete system of…
In this article we provide, from a novel perspective, arguments that support the idea that, in the wake of Cox' approach to probability theory, von Neumann's entropy should be the natural one in Quantum Mechanics. We also generalize the…