Related papers: Designing a Linearized Potential Function in Neura…

We derive a scaling property from a fundamental nonlinear differential equation whose solution is the so-called q-exponential function. A scaling property has been believed to be given by a power function only, but actually more general…

Gauss' law of error is generalized in Tsallis statistics such as multifractal systems, in which Tsallis entropy plays an essential role instead of Shannon entropy. For the generalization, we apply the new multiplication operation determined…

In this paper, we propose to quantitatively compare loss functions based on parameterized Tsallis-Havrda-Charvat entropy and classical Shannon entropy for the training of a deep network in the case of small datasets which are usually…

We study the properties of Tsallis entropy and Shannon entropy from the point of view of algorithmic randomness. In algorithmic information theory, there are two equivalent ways to define the program-size complexity K(s) of a given finite…

This article proposes a new two-parameter generalized entropy, which can be reduced to the Tsallis and the Shannon entropy for specific values of its parameters. We develop a number of information-theoretic properties of this generalized…

Improving generalization is one of the main challenges for training deep neural networks on classification tasks. In particular, a number of techniques have been proposed, aiming to boost the performance on unseen data: from standard data…

In this paper, we propose a new discriminative model named \emph{nonextensive information theoretical machine (NITM)} based on nonextensive generalization of Shannon information theory. In NITM, weight parameters are treated as random…

We propose a new way of defining entropy of a system, which gives a general form which may be nonextensive as Tsallis entropy, but is linearly dependent on component entropies, like Renyi entropy, which is extensive. This entropy has a…

We present a sampling-based trajectory optimization method derived from the maximum entropy formulation of Differential Dynamic Programming with Tsallis entropy. This method is a generalization of the legacy work with Shannon entropy, which…

Tsallis relative operator entropy was defined as a parametric extension of relative operator entropy and the generalized Shannon inequalities were shown in the previous paper. After the review of some fundamental properties of Tsallis…

Continuous normalizing flows (CNFs) are a generative method for learning probability distributions, which is based on ordinary differential equations. This method has shown remarkable empirical success across various applications, including…

The field of information science has greatly developed, and applications in various fields have emerged. In this paper, we evaluated the coding system in the theory of Tsallis entropy for transmission of messages and aimed to formulate the…

The Shannon capacity of a graph is an important graph invariant in information theory that is extremely difficult to compute. The Lovasz number, which is based on semidefinite programming relaxation, is a well-known upper bound for the…

We study the evolution of Tsallis entropy along the heat flow and establish its concavity in arbitrary dimensions. Extending prior results that were restricted to the one-dimensional setting, we prove that the Tsallis entropy is concave in…

In this paper we remark that Shannon entropy can be expressed as a function of the self-information (i.e. the logarithm) and the inverse of the Lambert $W$ function. It means that we consider that Shannon entropy has the trace form: $-k…

Deep learning utilizing deep neural networks (DNNs) has achieved a lot of success recently in many important areas such as computer vision, natural language processing, and recommendation systems. The lack of convexity for DNNs has been…

Estimating entropies from limited data series is known to be a non-trivial task. Naive estimations are plagued with both systematic (bias) and statistical errors. Here, we present a new 'balanced estimator' for entropy functionals Shannon,…

The coupled entropy is proven to correct a flaw in the derivation of the Tsallis entropy and thereby solidify the theoretical foundations for analyzing the uncertainty of complex systems. The Tsallis entropy originated from considering…

Even when massively overparameterized, deep neural networks show a remarkable ability to generalize. Research on this phenomenon has focused on generalization within distribution, via smooth interpolation. Yet in some settings neural…

In a recent paper [Phys. Lett. A {\bf335}, 351 (2005)] the authors discussed the equivalence among the various probability distribution functions of a system in equilibrium in the Tsallis entropy framework. In the present letter we extend…