相关论文: On the Practical Estimation and Interpretation of …

The R\'{e}nyi cross-entropy measure between two distributions, a generalization of the Shannon cross-entropy, was recently used as a loss function for the improved design of deep learning generative adversarial networks. In this work, we…

Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…

Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be…

One of the most influential results in neural network theory is the universal approximation theorem [1, 2, 3] which states that continuous functions can be approximated to within arbitrary accuracy by single-hidden-layer feedforward neural…

Deep reinforcement learning (RL) has shown remarkable success in complex domains, however, the inherent black box nature of deep neural network policies raises significant challenges in understanding and trusting the decision-making…

Inferring the parameters of a stochastic model based on experimental observations is central to the scientific method. A particularly challenging setting is when the model is strongly indeterminate, i.e. when distinct sets of parameters…

The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…

Mainstream Test-Time Adaptation (TTA) methods for adapting vision-language models, e.g., CLIP, typically rely on Shannon Entropy (SE) at test time to measure prediction uncertainty and inconsistency. However, since CLIP has a built-in bias…

We explore a large class of correlation measures called the $\alpha-z$ R\'enyi mutual informations (RMIs). Unlike the commonly used notion of RMI involving linear combinations of R\'enyi entropies, the $\alpha-z$ RMIs are positive…

A two-parameter family of complexity measures $\tilde{C}^{(\alpha,\beta)}$ based on the R\'enyi entropies is introduced and characterized by a detailed study of its mathematical properties. This family is the generalization of a continuous…

The equivalence between non-extensive C. Tsallis entropy and the extensive entropy introduced by Alfr\'ed R\'enyi is discussed. The R\'enyi entropy is studied from the perspective of the geometry of the Lebesgue and generalised, exotic…

R\'enyi entropies are conceptually valuable and experimentally relevant generalisations of the celebrated von Neumann entanglement entropy. After a quantum quench in a clean quantum many-body system they generically display a universal…

Subword tokenization is a key part of many NLP pipelines. However, little is known about why some tokenizer and hyperparameter combinations lead to better downstream model performance than others. We propose that good tokenizers lead to…

We revisit the problem of asymmetric binary hypothesis testing against a composite alternative hypothesis. We introduce a general framework to treat such problems when the alternative hypothesis adheres to certain axioms. In this case we…

The asymptotic restriction problem for tensors can be reduced to finding all parameters that are normalized, monotone under restrictions, additive under direct sums and multiplicative under tensor products, the simplest of which are the…

Test-time adaptation (TTA) aims to transfer knowledge from a source model to unknown test data with potential distribution shifts in an online manner. Many existing TTA methods rely on entropy as a confidence metric to optimize the model.…

The R\'enyi-Shannon entropy associated to critical quantum spins chain with central charge $c=1$ is shown to have a phase transition at some value $n_c$ of the R\'enyi parameter $n$ which depends on the Luttinger parameter (or…

A general investigation is made into the intrinsic Riemannian geometry for complex systems, from the perspective of statistical mechanics. The entropic formulation of statistical mechanics is the ingredient which enables a connection…

We revisit generalized entropic formulations of the uncertainty principle for an arbitrary pair of quantum observables in two-dimensional Hilbert space. R\'enyi entropy is used as uncertainty measure associated with the distribution…

This paper introduces "swiveled Renyi entropies" as an alternative to the Renyi entropic quantities put forward in [Berta et al., Phys. Rev. A 91, 022333 (2015)]. What distinguishes the swiveled Renyi entropies from the prior proposal of…