Related papers: Multivariate Extension of Matrix-based Renyi's \al…
R\'enyi transfer entropy (RTE) is a generalization of classical transfer entropy that replaces Shannon's entropy with R\'enyi's information measure. This, in turn, introduces a new tunable parameter $\alpha$, which accounts for sensitivity…
Reconstructing the structural connectivity between interacting units from observed activity is a challenge across many different disciplines. The fundamental first step is to establish whether or to what extent the interactions between the…
Numerous entropy-type characteristics (functionals) generalizing R\'enyi entropy are widely used in mathematical statistics, physics, information theory, and signal processing for characterizing uncertainty in probability distributions and…
Multidimensional function data arise from many fields nowadays. The covariance function plays an important role in the analysis of such increasingly common data. In this paper, we propose a novel nonparametric covariance function estimation…
Entanglement entropy in even dimensional conformal field theories (CFTs) contains well-known universal terms arising from the conformal anomaly. Renyi entropies are natural generalizations of the entanglement entropy that are much less…
We study different aspects of quantum field theory at finite density using methods from quantum information theory. For simplicity we focus on massive Dirac fermions with nonzero chemical potential, and work in $1+1$ space-time dimensions.…
The interactions between three or more random variables are often nontrivial, poorly understood, and yet, are paramount for future advances in fields such as network information theory, neuroscience, genetics and many others. In this work,…
Information-theoretic quantities like entropy and mutual information have found numerous uses in machine learning. It is well known that there is a strong connection between these entropic quantities and submodularity since entropy over a…
Feature selection aims to select the smallest feature subset that yields the minimum generalization error. In the rich literature in feature selection, information theory-based approaches seek a subset of features such that the mutual…
Theoretical studies have proven that the Hilbert space has remarkable performance in many fields of applications. Frames in tensor product of Hilbert spaces were introduced to generalize the inner product to high-order tensors. However,…
Conditional mutual information is important in the selection and interpretation of graphical models. Its empirical version is well known as a generalised likelihood ratio test and that it may be represented as a difference in entropy. We…
We propose R\'enyi information generating function and discuss its properties. A connection between the R\'enyi information generating function and the diversity index is proposed for discrete type random variables. The relation between the…
An uncertainty relation for the R\'enyi entropies of conjugate quantum observables is used to obtain a strong Heisenberg limit of the form ${\rm RMSE} \geq f(\alpha)/(\langle N\rangle+\frac12)$, bounding the root mean square error of any…
The concept of entropy connects the number of possible configurations with the number of variables in large stochastic systems. Independent or weakly interacting variables render the number of configurations scale exponentially with the…
Using arguments built on ergodicity, we derive an analytical expression for the Renyi entanglement entropies corresponding to the finite-energy density eigenstates of chaotic many-body Hamiltonians. The expression is a universal function of…
The concept of entropy, firstly introduced in information theory, rapidly became popular in many applied sciences via Shannon's formula to measure the degree of heterogeneity among observations. A rather recent research field aims at…
We introduce a general and simple structural design called Multiplicative Integration (MI) to improve recurrent neural networks (RNNs). MI changes the way in which information from difference sources flows and is integrated in the…
Taking into account extremum of a Helmholtz free energy in the equilibrium state of a thermodynamic system the Renyi entropy is derived from the Boltzmann entropy by the same way as the Helmholtz free energy from the Hamiltonian. The…
We propose an extension of the sandwiched R\'enyi relative $\alpha$-entropy to normal positive functionals on arbitrary von Neumann algebras, for the values $\alpha>1$. For this, we use Kosaki's definition of noncommutative $L_p$-spaces…
One of the most influential results in neural network theory is the universal approximation theorem [1, 2, 3] which states that continuous functions can be approximated to within arbitrary accuracy by single-hidden-layer feedforward neural…