Related papers: R\'enyi divergence-based uniformity guarantees for…
We prove several universal approximation results at minimal or near-minimal width for approximation of $L^p(\mathbb{R}^{d_x}, \mathbb{R}^{d_y})$ and $C^0(\mathbb{R}^{d_x}, \mathbb{R}^{d_y})$ on compact sets. Our approach uses a unified…
Binary hashing is a well-known approach for fast approximate nearest-neighbor search in information retrieval. Much work has focused on affinity-based objective functions involving the hash functions or binary codes. These objective…
Entropies must correspond to mean values for them to be measurable. The Shannon entropy corresponds to the weighted arithmetic mean, whereas the Renyi entropy corresponds to the exponential mean. These means refer to code lengths, which are…
The concept of uniform distribution in $[0,1]$ is extended for a certain strictly separated maximal (in the sense of cardinality) family $(\lambda_t)_{t \in [0,1]}$ of invariant extensions of the linear Lebesgue measure $\lambda$ in…
Renyi's information measures ---the Renyi information, mean, capacity, radius, and center--- are analyzed relying on the elementary properties of the Renyi divergence and the power means. The van Erven-Harremoes conjecture is proved for any…
We present conditions that allow us to pass from the convergence of probability measures in distribution to the uniform convergence of the associated quantile functions. Under these conditions, one can in particular pass from the asymptotic…
Motivated from the fact that universal source coding on countably infinite alphabets is not feasible, this work introduces the notion of almost lossless source coding. Analog to the weak variable-length source coding problem studied by Han…
The matrix-based Renyi's \alpha-order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the current theory in the…
Many axiomatic definitions of entropy, such as the R\'enyi entropy, of a random variable are closely related to the $\ell_{\alpha}$-norm of its probability distribution. This study considers probability distributions on finite sets, and…
On a compact group the Haar probability measure plays the role of uniform distribution. The entropy and rate distortion theory for this uniform distribution is studied. New results and simplified proofs on convergence of convolutions on…
Deep hashing enables image retrieval by end-to-end learning of deep representations and hash codes from training data with pairwise similarity information. Subject to the distribution skewness underlying the similarity information, most…
We study the best approximation problem: \[ \displaystyle \min_{\alpha\in \mathbb R^m}\max_{1\leq i\leq n}\left|y_i -\sum_{j=1}^m \alpha_j \Gamma_j ({\bf x}_i) \right|. \] Here: $\Gamma:=\left\{\Gamma_1,...,\Gamma_m\right\}$ is a list of…
We propose a non-parametric anomaly detection algorithm for high dimensional data. We score each datapoint by its average $K$-NN distance, and rank them accordingly. We then train limited complexity models to imitate these scores based on…
We consider a two-parameter family of R\'enyi relative entropies $D_{\alpha,z}(\rho||\sigma)$ that are quantum generalisations of the classical R\'enyi divergence $D_{\alpha}(p||q)$. This family includes many known relative entropies (or…
In this paper, we prove a universality result for the limiting distribution of persistence diagrams arising from geometric filtrations over random point processes. Specifically, we consider the distribution of the ratio of persistence…
We provide the sandwiched R\'enyi divergence of order $\alpha\in(\frac{1}{2},1)$, as well as its induced quantum information quantities, with an operational interpretation in the characterization of the exact strong converse exponents of…
Quantum key distribution requires tight and reliable bounds on the secret key rate to ensure robust security. This is particularly so for the regime of finite block sizes, where the optimization of generalized R\'enyi entropic quantities is…
Given a length $n$ sample from $\mathbb{R}^d$ and a neural network with a fixed architecture with $W$ weights, $k$ neurons, linear threshold activation functions, and binary outputs on each neuron, we study the problem of uniformly sampling…
Quantum generalizations of R\'enyi's entropies are a useful tool to describe a variety of operational tasks in quantum information processing. Two families of such generalizations turn out to be particularly useful: the Petz quantum R\'enyi…
We consider the random lasing from a weakly scattering medium and demonstrate that the distribution of the threshold gain over the ensemble of statistically independent finite-size samples is universal. Universality stems from the facts…