Related papers: A new definition of t-entropy for transfer operato…
Tensor transpose is a higher order generalization of matrix transpose. In this paper, we use permutations and symmetry group to define? the tensor transpose. Then we discuss the classification and composition of tensor transposes.…
The translation of written language has been known since the 3rd century BC; however, its necessity has become increasingly common in the information age. Today, many translators exist, based on encoder-decoder deep architectures,…
An efficient approach to the calculation of the $\epsilon$-entropy is proposed. The method is based on the idea of looking at the information content of a string of data, by analyzing the signal only at the instants when the fluctuations…
Entropy is a very useful concept from physics that tries to explain how a system behaves from a point of view of the thermodynamics. However, there are two ways to explain entropy, and it depends on if we are studying a microsystem or a…
A brief discussion is given of the traditional version of the Maximum Entropy Method, including a review of some of the criticism that has been made in regard to its use in statistical inference. Motivated by these questions, a modified…
A general black-branes system under the T-duality transformation will become another smeared system with different dimensional black branes. We first use some simple examples to see that both systems have a same value of entropy and then…
The R\'{e}nyi cross-entropy measure between two distributions, a generalization of the Shannon cross-entropy, was recently used as a loss function for the improved design of deep learning generative adversarial networks. In this work, we…
This letter reports two moment extensions of the entropy of a distribution. By understanding the traditional entropy as the average of the original distribution up to a random variable transformation, the traditional moments equation become…
We introduce the concept of inflation word entropy for random substitutions with a constant and primitive substitution matrix. Previous calculations of the topological entropy of such systems implicitly used this concept and established…
We introduce a technique to estimate a linear operator by embedding it in a family $A_t$ of operators, $t\in(\sigma_0,\infty)$, with suitable curvature properties. One can then estimate the norm of each $A_t$ by bounds that hold in the…
In this paper, we present a new class of operators, which we name to be $n$-Ritt operators. This produces a discrete analogue of $n$-sectorial operators and generalizes the notion of Ritt operators. We develop a $H^\infty$-functional…
We define the entropy operator as the negative of the logarithm of the density matrix, give a prescription for extracting its thermodynamically measurable part, and discuss its dynamics. For an isolated system we derive the first, second…
The definition of weighted entropy allows for easy calculation of the entropy of the mixture of measures. In this paper we investigate the problem of equivalent definition of the general entropy function in weighted form. We show that under…
Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…
A notion of entropy is introduced for causal fermion systems. This entropy is a measure of the state of disorder of a causal fermion system at a given time compared to the vacuum. The definition is given both in the finite and…
A definition of entropy via the Kolmogorov algorithmic complexity is discussed. As examples, we show how the meanfield theory for the Ising model, and the entropy of a perfect gas can be recovered. The connection with computations are…
Next-token prediction is conventionally done using decoder-only Transformers with causal attention, as this approach allows for efficient reuse of keys and values. What if we were not compute-limited, should we still use decoder-only…
One critical component in lossy deep image compression is the entropy model, which predicts the probability distribution of the quantized latent representation in the encoding and decoding modules. Previous works build entropy models upon…
Transfer entropy (TE) is a popular measure of information flow found to perform consistently well in different settings. Symbolic transfer entropy (STE) is defined similarly to TE but on the ranks of the components of the reconstructed…
With the help of transfer entropy, we analyze information flows between communities of complex networks. We show that the transfer entropy provides a coherent description of interactions between communities, including non-linear…