Related papers: Numerical Stability of Generalized Entropies
The paper examines relationships between the conditional Shannon entropy and the expectation of $\ell_{\alpha}$-norm for joint probability distributions. More precisely, we investigate the tight bounds of the expectation of…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…
Recently, information theoretic analysis has become a popular framework for understanding the generalization behavior of deep neural networks. It allows a direct analysis for stochastic gradient/Langevin descent (SGD/SGLD) learning…
We investigate the cumulative Tsallis entropy, an information measure recently introduced as a cumulative version of the classical Tsallis differential entropy, which is itself a generalization of the Boltzmann-Gibbs statistics. This…
For certain families of finite quantum graphs, we study the question of how eigenfunctions are distributed over the graph. To characterize properties of the distribution, generalized entropies of the R\'{e}nyi and Tsallis types are…
Based on the Tsallis entropy, the nonextensive thermodynamic properties are studied as a q-deformation of classical statistical results using only probabilistic methods and straightforward calculations. It is shown that the constant in the…
We derive entropic uncertainty relations for successive generalized measurements by using general descriptions of quantum measurement within two {distinctive operational} scenarios. In the first scenario, by merging {two successive…
We provide a unifying axiomatics for Renyi's entropy and non-extensive entropy of Tsallis. It is shown that the resulting entropy coincides with Csiszar's measure of directed divergence known from communication theory.
We explore the relation between entanglement entropy of quantum many body systems and the distribution of corresponding, properly selected, observables. Such a relation is necessary to actually measure the entanglement entropy. We show that…
It is known that state-dependent, multi-step Lyapunov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the "fluid model" approach to stability of…
Few parameters dependent generalised entropy includes Tsallis entropy, R{\'e}nyi entropy, Sharma-Mittal entropy, Barrow entropy, Kaniadakis entropy, etc as particular representatives. Its relation to physical systems is not always clear. In…
We describe in detail two numerical simulation methods valid to study systems whose thermostatistics is described by generalized entropies, such as Tsallis. The methods are useful for applications to non-trivial interacting systems with a…
We show how generalized Gibbs-Shannon entropies can provide new insights on the statistical properties of texts. The universal distribution of word frequencies (Zipf's law) implies that the generalized entropies, computed at the word level,…
In this paper we give an interpretation of Tsallis' nonextensive statistical mechanics based upon the information-theoretic point of view of Luzzi et al. [cond-mat/0306217; cond-mat/0306247; cond-mat/0307325], suggesting Tsallis' entropy to…
We show that different entropic measures of fluctuations lead to contradictory uncertainty relations for two complementary observables. We apply Tsallis and R\'{e}nyi entropies to the joint distribution emerging from a noisy simultaneous…
The aim of the paper is to study the link between non additivity of some entropies and their boundedness. We propose an axiomatic construction of the entropy relying on the fact that entropy belongs to a group isomorphic to the usual…
The requirement that an entropy function be composable is key: it means that the entropy of a compound system can be calculated in terms of the entropy of its independent components. We prove that, under mild regularity assumptions, the…
This is full length article (draft version) where problem number of topics in Topic Modeling is discussed. We proposed idea that Renyi and Tsallis entropy can be used for identification of optimal number in large textual collections. We…
Estimation of Shannon and R\'enyi entropies of unknown discrete distributions is a fundamental problem in statistical property testing and an active research topic in both theoretical computer science and information theory. Tight bounds on…
Entropy notions for $\varepsilon$-incremental practical stability and incremental stability of deterministic nonlinear systems under disturbances are introduced. The entropy notions are constructed via a set of points in state space which…