Related papers: Arimoto-R\'enyi Conditional Entropy and Bayesian $…
The min-entropy is a widely used metric to quantify the randomness of generated random numbers in cryptographic applications; it measures the difficulty of guessing the most likely output. An important min-entropy estimator is the…
For a system described by a multivariate probability density function obeying the fluctuation theorem, the average dissipation is lower-bounded by the degree of asymmetry of the marginal distributions (namely the relative entropy between…
This paper shows that the logarithm of the epsilon-error capacity (average error probability) for n uses of a discrete memoryless channel is upper bounded by the normal approximation plus a third-order term that does not exceed 1/2 log n +…
A task is randomly drawn from a finite set of tasks and is described using a fixed number of bits. All the tasks that share its description must be performed. Upper and lower bounds on the minimum $\rho$-th moment of the number of performed…
A novel definition of the conditional smooth Renyi entropy, which is different from that of Renner and Wolf, is introduced. It is shown that our definition of the conditional smooth Renyi entropy is appropriate to give lower and upper…
Sharpness (of the loss minima) is a common measure to investigate the generalization of neural networks. Intuitively speaking, the flatter the landscape near the minima is, the better generalization might be. Unfortunately, the correlation…
We propose an extension of the classical R\'enyi divergences to quantum states through an optimization over probability distributions induced by restricted sets of measurements. In particular, we define the notion of locally-measured…
In this work, maximal $\alpha$-leakage is introduced to quantify how much a quantum adversary can learn about any sensitive information of data upon observing its disturbed version via a quantum privacy mechanism. We first show that an…
New non-asymptotic random coding theorems (with error probability $\epsilon$ and finite block length $n$) based on Gallager parity check ensemble and Shannon random code ensemble with a fixed codeword type are established for discrete input…
The paper focuses on general properties of parametric minimum contrast estimators. The quality of estimation is measured in terms of the rate function related to the contrast, thus allowing to derive exponential risk bounds invariant with…
Given a set of discrete probability distributions, the minimum entropy coupling is the minimum entropy joint distribution that has the input distributions as its marginals. This has immediate relevance to tasks such as entropic causal…
We extend the notion of estimation entropy of autonomous dynamical systems proposed by Liberzon and Mitra [1] to nonlinear dynamical systems with uncertain inputs with bounded variation. We call this new notion the {$\epsilon$}-estimation…
The paper examines relationships between the conditional Shannon entropy and the expectation of $\ell_{\alpha}$-norm for joint probability distributions. More precisely, we investigate the tight bounds of the expectation of…
We prove a lower estimate on the increase in entropy when two copies of a conditional random variable $X | Y$, with $X$ supported on $\mathbb{Z}_q=\{0,1,\dots,q-1\}$ for prime $q$, are summed modulo $q$. Specifically, given two i.i.d copies…
We provide an exact non-asymptotic lower bound on the minimax expected excess risk (EER) in the agnostic probably-ap\-proximately-correct (PAC) machine learning classification model and identify minimax learning algorithms as certain…
We consider the problem of binary classification with abstention in the relatively less studied \emph{bounded-rate} setting. We begin by obtaining a characterization of the Bayes optimal classifier for an arbitrary input-label distribution…
The entropy of probability distribution defined by Shannon has several extensions. R\'enyi entropy is one of the general extensions of Shannon entropy and is widely used in engineering, physics, and so on. On the other hand, the quantum…
We express the joint R\'enyi entropy of progressively censored order statistics in terms of an incomplete integral of the hazard function, and provide a simple estimate of the joint R\'enyi entropy of progressively Type-II censored data.…
We show that the minimum Renyi entropy output of a quantum channel is locally additive for Renyi parameter alpha>1. While our work extends the results of [10] (in which local additivity was proven for alpha=1), it is based on several new…
The Renyi entropies constitute a family of information measures that generalizes the well-known Shannon entropy, inheriting many of its properties. They appear in the form of unconditional and conditional entropies, relative entropies or…