Related papers: On the Conditional Smooth Renyi Entropy and its Ap…
A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the…
This paper establishes the exact strong converse exponent of the soft covering problem in the classical setting. This exponent characterizes the slowest achievable convergence speed of the total variation to one when a code of rate below…
The Boltzmann entropy $S^{(B)}$ is true in the case of equal probability of all microstates of a system. In the opposite case it should be averaged over all microstates that gives rise to the Boltzmann--Shannon entropy (BSE). Maximum…
Maximal repetition of a string is the maximal length of a repeated substring. This paper investigates maximal repetition of strings drawn from stochastic processes. Strengthening previous results, two new bounds for the almost sure growth…
In the present follow-up article of a previous one [1] we illustrate the use of the Unconventional Statistical Mechanics described and discussed in the latter. This is done via the analysis, resorting to Renyi approach, of experimental…
Entropy is a fundamental concept in equilibrium statistical mechanics, yet its origin in the non-equilibrium dynamics of isolated quantum systems is not fully understood. A strong consensus is emerging around the idea that the stationary…
Recent improvements in the method of estimating Renyi entropies from measurements of coincidences between the events observed in high energy collisions are reviewed. A new, more precise, formulation of the method is presented and its…
A new upper bound on the relative entropy is derived as a function of the total variation distance for probability measures defined on a common finite alphabet. The bound improves a previously reported bound by Csisz\'ar and Talata. It is…
High level declarative constraints provide a powerful (and popular) way to define and construct control policies; however, most synthesis algorithms do not support specifying the degree of randomness (unpredictability) of the resulting…
We prove that the average smooth Renyi entropy rate will approach the entropy rate of a stationary, ergodic information source, which is equal to the Shannon entropy rate for a classical information source and the von Neumann entropy rate…
The minimum error entropy (MEE) criterion has been verified as a powerful approach for non-Gaussian signal processing and robust machine learning. However, the implementation of MEE on robust classification is rather a vacancy in the…
The new estimates of the conditional Shannon entropy are introduced in the framework of the model describing a discrete response variable depending on a vector of d factors having a density w.r.t. the Lebesgue measure in R^d. Namely, the…
This paper establishes a close relationship among the four information theoretic problems, namely Campbell source coding, Arikan guessing, Huleihel et al. memoryless guessing and Bunte and Lapidoth tasks partitioning problems. We first show…
A task is randomly drawn from a finite set of tasks and is described using a fixed number of bits. All the tasks that share its description must be performed. Upper and lower bounds on the minimum $\rho$-th moment of the number of performed…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
This paper starts by considering the minimization of the Renyi divergence subject to a constraint on the total variation distance. Based on the solution of this optimization problem, the exact locus of the points $\bigl( D(Q\|P_1),…
Most of the existing classification methods are aimed at minimization of empirical risk (through some simple point-based error measured with loss function) with added regularization. We propose to approach this problem in a more information…
This thesis consolidates, improves and extends the smooth entropy framework for non-asymptotic information theory and cryptography. We investigate the conditional min- and max-entropy for quantum states, generalizations of classical R\'enyi…
Recently a new quantum generalization of the Renyi divergence and the corresponding conditional Renyi entropies was proposed. Here we report on a surprising relation between conditional Renyi entropies based on this new generalization and…
The Random Permutation Set (RPS) is a new type of set proposed recently, which can be regarded as the generalization of evidence theory. To measure the uncertainty of RPS, the entropy of RPS and its corresponding maximum entropy have been…