Related papers: Codes for Tasks and R\'enyi Entropy Rate
The recently developed matrix based Renyi's entropy enables measurement of information in data simply using the eigenspectrum of symmetric positive semi definite (PSD) matrices in reproducing kernel Hilbert space, without estimation of the…
A pattern of a sequence is a sequence of integer indices with each index describing the order of first occurrence of the respective symbol in the original sequence. In a recent paper, tight general bounds on the block entropy of patterns of…
The R\'enyi entropy is a mathematical generalization of the concept of entropy and it encodes the total information of a system as a funtion of its order parameter $\alpha$. The meaning of the R\'enyi entropy in physics is not completely…
How hard is it guess a password? Massey showed that that the Shannon entropy of the distribution from which the password is selected is a lower bound on the expected number of guesses, but one which is not tight in general. In a series of…
Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order…
The asymptotic restriction problem for tensors can be reduced to finding all parameters that are normalized, monotone under restrictions, additive under direct sums and multiplicative under tensor products, the simplest of which are the…
The R{\'e}nyi entropy is one of the important information measures that generalizes Shannon's entropy. The quantum R{\'e}nyi entropy has a fundamental role in quantum information theory, therefore, bounding this quantity is of vital…
We investigate the longest common substring problem for encoded sequences and its asymptotic behaviour. The main result is a strong law of large numbers for a re-scaled version of this quantity, which presents an explicit relation with the…
Information theory on a time-discrete setting in the framework of time series analysis is generalized to the time-continuous case. Considerations of the Roessler and Lorenz dynamics as well as the Ornstein-Uhlenbeck process yield for…
We consider a probability distribution depending on a real parameter $x$. As functions of $x$, the R\'enyi entropy and the Tsallis entropy can be expressed in terms of the associated index of coincidence $S(x)$. We establish recurrence…
Min-entropy sampling gives a bound on the min-entropy of a randomly chosen subset of a string, given a bound on the min-entropy of the whole string. K\"onig and Renner showed a min-entropy sampling theorem that holds relative to quantum…
Time-reversal symmetry breaking and entropy production are universal features of nonequilibrium phenomena. Despite its importance in the physics of active and living systems, the entropy production of systems with many degrees of freedom…
In this paper we study certain properties of R\'{e}nyi entropy functionals $H_\alpha(\mathcal{P})$ on the space of probability distributions over $\mathbb{Z}_+$. Primarily, continuity and convergence issues are addressed. Some properties…
Relative entropy is a measure of distinguishability for quantum states, and plays a central role in quantum information theory. The family of Renyi entropies generalizes to Renyi relative entropies that include as special cases most entropy…
Maximal repetition of a string is the maximal length of a repeated substring. This paper investigates maximal repetition of strings drawn from stochastic processes. Strengthening previous results, two new bounds for the almost sure growth…
It is well known that the entropy $H(X)$ of a discrete random variable $X$ is always greater than or equal to the entropy $H(f(X))$ of a function $f$ of $X$, with equality if and only if $f$ is one-to-one. In this paper, we give tight…
A novel definition of the conditional smooth Renyi entropy, which is different from that of Renner and Wolf, is introduced. It is shown that our definition of the conditional smooth Renyi entropy is appropriate to give lower and upper…
For a closed-loop control system with a digital channel between the sensor and the controller, the notion of invariance entropy quantifies the smallest average rate of information above which a given compact subset of the state space can be…
Distributions of abundances or frequencies play an important role in many fields of science, from biology to sociology, as does the R\'enyi entropy, which measures the diversity of a statistical ensemble. We derive a mathematical relation…
R\'enyi entropy is an important measure in the context of information theory as a generalization of Shannon entropy. This information measure was often used for uncertainty quantification of dynamical behaviour of stochastic processes. In…