Related papers: Submodular Combinatorial Information Measures with…
Entropic uncertainty relations are interesting in their own rights as well as for a lot of applications. Keeping this in mind, we try to make the corresponding inequalities as tight as possible. The use of parametrized entropies also allows…
We introduce and study the cumulative information generating function, which provides a unifying mathematical tool suitable to deal with classical and fractional entropies based on the cumulative distribution function and on the survival…
Estimating mutual information (MI) from samples is a fundamental problem in statistics, machine learning, and data analysis. Recently it was shown that a popular class of non-parametric MI estimators perform very poorly for strongly…
The mutual information is a core statistical quantity that has applications in all areas of machine learning, whether this is in training of density models over multiple data modalities, in maximising the efficiency of noisy transmission…
Imitation learning holds the promise of equipping robots with versatile skills by learning from expert demonstrations. However, policies trained on finite datasets often struggle to generalize beyond the training distribution. In this work,…
The mutual information method has demonstrated to be very useful for deriving the potential order parameter of a system. Although the method suggests some constraints which help to define this quantity, there is still some freedom in the…
A new measure of information in quantum mechanics is proposed which takes into account that for quantum systems the only feature known before an experiment is performed are the probabilities for various events to occur. The sum of the…
A novel measure, quantumness of correlations is introduced here for bipartite states, by incorporating the required measurement scheme crucial in defining any such quantity. Quantumness coincides with the previously proposed measures in…
We pedagogically present the information theory as originally established, explaining its essential ideas and paying attention to the expression employed to measure the amount of information. Also we discussed relationships between…
One of the fundamental steps toward understanding a complex system is identifying variation at the scale of the system's components that is most relevant to behavior on a macroscopic scale. Mutual information provides a natural means of…
Robust optimization is becoming increasingly important in machine learning applications. In this paper, we study a unified framework of robust submodular optimization. We study this problem both from a minimization and maximization…
Using the relative entropy of total correlation, we derive an expression relating the mutual information of $n$-partite pure states to the sum of the mutual informations and entropies of its marginals and analyze some of its implications.…
Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order…
We discuss the connection between information and copula theories by showing that a copula can be employed to decompose the information content of a multivariate distribution into marginal and dependence components, with the latter…
Motivated by thermodynamic considerations, we analyse the variation of the quantum mutual information on a unitary orbit of a bipartite system's state, with and without global constraints such as energy conservation. We solve the full…
Observational entropy is interpreted as the uncertainty an observer making measurements associates with a system. So far, properties that make such an interpretation possible rely on the assumption of ideal projective measurements. We show…
The so-called {\em leakage-chain rule} is a very important tool used in many security proofs. It gives an upper bound on the entropy loss of a random variable $X$ in case the adversary who having already learned some random variables…
A variety of new measures of quantum Renyi mutual information and quantum Renyi conditional entropy have recently been proposed, and some of their mathematical properties explored. Here, we show that the Renyi mutual information attains…
Analysis of a probabilistic system often requires to learn the joint probability distribution of its random variables. The computation of the exact distribution is usually an exhaustive precise analysis on all executions of the system. To…
The concepts of conditional mutual information (CMI) and normalized conditional mutual information (NCMI) are introduced to measure the concentration and separation performance of a classification deep neural network (DNN) in the output…