Related papers: Nonextensive information theoretical machine
Probability distributions defined on the half space are known to be quite different from those in the full space. Here, a nonextensive entropic treatment is presented for the half space in an analytic and self-consistent way. In this…
Using established principles from Statistics and Information Theory, we show that invariance to nuisance factors in a deep neural network is equivalent to information minimality of the learned representation, and that stacking layers and…
We describe how to analyze the wide class of non stationary processes with stationary centered increments using Shannon information theory. To do so, we use a practical viewpoint and define ersatz quantities from time-averaged probability…
Recently, a new measure of information called extropy has been introduced by Lad, Sanfilippo and Agr\`o as the dual version of Shannon entropy. In the literature, Tsallis introduced a measure for a discrete random variable, named Tsallis…
The influence of the dimerization process on the nematic ordering is investigated by using a nonextensive thermostatistics, namely Tsallis thermostatistics(TT). A theoretical model taking into account the dimerization influence on the…
This article proposes a new two-parameter generalized entropy, which can be reduced to the Tsallis and the Shannon entropy for specific values of its parameters. We develop a number of information-theoretic properties of this generalized…
Examples of joint probability distributions are studied in terms of Tsallis' nonextensive statistics both for correlated and uncorrelated variables, in particular it is explicitely shown how correlations in the system can make Tsallis…
The formalism of statistical mechanics can be generalized by starting from more general measures of information than the Shannon entropy and maximizing those subject to suitable constraints. We discuss some of the most important examples of…
Probability distributions which emerge from the formalism of nonextensive statistical mechanics have been applied to a variety of problems. In this paper we unite modeling of such distributions with the model of widespread 1/f noise. We…
This paper proposes a new principled multi-task representation learning framework (InfoMTL) to extract noise-invariant sufficient representations for all tasks. It ensures sufficiency of shared representations for all tasks and mitigates…
A class of estimators of the R\'{e}nyi and Tsallis entropies of an unknown distribution $f$ in $\mathbb{R}^m$ is presented. These estimators are based on the $k$th nearest-neighbor distances computed from a sample of $N$ i.i.d. vectors with…
Quantifying the influence of infinitesimal changes in training data on model performance is crucial for understanding and improving machine learning models. In this work, we reformulate this problem as a weighted empirical risk minimization…
In this paper, we investigate the asymptotic stability of finite-dimensional stochastic integrable Hamiltonian systems via information entropy. Specifically, we establish the asymptotic vanishing of Shannon entropy difference (with…
Tsallis entropy is a generalized diversity index first derived in Patil and Taillie (1982) and then rediscovered in community ecology by Keylock (2005). Bayesian nonparametric estimation of Shannon entropy and Simpson's diversity under…
The nonextensive statistics based on Tsallis entropy have been so far used for the systems composed of subsystems having same $q$. The applicability of this statistics to the systems with different $q$'s is still a matter of investigation.…
Nonextensive Statistical Mechanics has developed into an important framework for modeling the thermodynamics of complex systems and the information of complex signals. Upon the 80th birthday of the field's founder, Constantino Tsallis, a…
This paper studies Fenchel-Young losses, a generic way to construct convex loss functions from a regularization function. We analyze their properties in depth, showing that they unify many well-known loss functions and allow to create…
In this paper, we have analyzed the nonextensive Tsallis statistical mechanics in the light of Verlinde's formalism. We have obtained, with the aid of a noncommutative phase-space entropic gravity, a new bound for Tsallis nonextensive (NE)…
There has been a significant recent surge in deep neural network (DNN) techniques. Most of the existing DNN techniques have restricted model formats/assumptions. To overcome their limitations, we propose the nonparametric transformation…
Information theory provides tools to predict the performance of a learning algorithm on a given dataset. For instance, the accuracy of learning an unknown parameter can be upper bounded by reducing the learning task to hypothesis testing…