Related papers: Some results concerning maximum Renyi entropy dist…
Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, $S=-\sum_i p_i \log p_i$. Here we note that in physical systems one of these axioms may be violated. For non-ergodic…
We consider the analysis of probability distributions through their associated covariance operators from reproducing kernel Hilbert spaces. We show that the von Neumann entropy and relative entropy of these operators are intimately related…
Estimation of Shannon and R\'enyi entropies of unknown discrete distributions is a fundamental problem in statistical property testing and an active research topic in both theoretical computer science and information theory. Tight bounds on…
A unified thermodynamic formalism describing the efficiency of learning is proposed. First, we derive an inequality, which is more strength than Clausius's inequality, revealing the lower bound of the entropy-production rate of a subsystem.…
The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example.…
We use a R\'enyi entropy method to prove strong converse theorems for certain information-theoretic tasks which involve local operations and quantum or classical communication between two parties. These include state redistribution,…
Recently, Suyari has proposed a generalization of Shannon-Khinchin axioms, which determines a class of entropies containing the well-known Tsalis and Havrda-Charvat entropies [H. Suyari, IEEE Trans. Inf. Theory, vol. 50, pp. 1783-1787, Aug.…
Statistical Physics, Diffusion Entropy Analysis and Information Theory commonly use Mathai's entropy which measures the randomness of probability laws, whereas welfare economics and the Social Sciences commonly use Gini index which measures…
We investigate the tightness and optimality of thermodynamic-uncertainty-relation (TUR)-type inequalities from two aspects, the choice of the Fisher information and the class of possible observables. We show that there exists the best…
Shannon-Renyi and stabilizer entropies are key diagnostics of structure, non-stabilizerness, phase transitions, and universality in quantum many-body states. We establish an exact correspondence for quadratic fermions: for any nondegenerate…
The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if…
We introduce a constructive framework for assigning thermodynamic structure to an arbitrary data system from its measured microstates. Starting from an empirical distribution over configurations, we first infer a data-driven energy function…
Recent literature in the last Maximum Entropy workshop introduced an analogy between cumulative probability distributions and normalized utility functions. Based on this analogy, a utility density function can de defined as the derivative…
We introduce some new classes of unimodal rotational invariant directional distributions, which generalize von Mises-Fisher distribution. We propose three types of distributions, one of which represents axial data. For each new type we…
The phenomenon of entropy concentration provides strong support for the maximum entropy method, MaxEnt, for inferring a probability vector from information in the form of constraints. Here we extend this phenomenon, in a discrete setting,…
We establish several convexity properties for the entropy and Fisher information of mixtures of centered Gaussian distributions. First, we prove that if $X_1, X_2$ are independent scalar Gaussian mixtures, then the entropy of $\sqrt{t}X_1 +…
Relative to the Gaussian measure on $\mathbb{R}^d$, entropy and Fisher information are famously related via Gross' logarithmic Sobolev inequality (LSI). These same functionals also separately satisfy convolution inequalities, as proved by…
In this paper, we prove the anisotropic Shannon inequality for the Renyi entropy with the best constant on Folland-Stein homogeneous Lie groups. As a consequence, we also prove the optimal Shannon inequality in the same setting. Using a…
In this paper, we provide the R\'enyi entropy and complexity measure for a novel, flexible class of skew-gaussian distributions and their related families, as a characteristic form of the skew-gaussian Shannon entropy. We give closed…
Using R\'enyi entropy, a possible thermostatistics for nonextensive systems is discussed. We show that it is possible to get the $q$-exponential distribution function for nonextensive systems having nonadditive energy but additive entropy.…