Related papers: Entropy and Long range correlations in literary En…
Maximal repetition of a string is the maximal length of a repeated substring. This paper investigates maximal repetition of strings drawn from stochastic processes. Strengthening previous results, two new bounds for the almost sure growth…
This paper aims to show how some popular topics on social networks can be used to predict online newspaper views, related to the topics. Newspapers site and many social networks, become a good source of data to analyse and explain complex…
Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over…
Languages across the world exhibit Zipf's law of abbreviation, namely more frequent words tend to be shorter. The generalized version of the law - an inverse relationship between the frequency of a unit and its magnitude - holds also for…
The derivation of the maximum entropy distribution of particles in boxes yields two kinds of distributions: a "bell-like" distribution and a long-tail distribution. The first one is obtained when the ratio between particles and boxes is…
In a recent letter (PRL 95, 190601 (2005)), T.M. Rocha Filho and coworkers address the very interesting issue of the entropic form to be used for Hamiltonians with long-range interactions. In our opinion the letter misses several points…
Zipf's law on word frequency is observed in English, French, Spanish, Italian, and so on, yet it does not hold for Chinese, Japanese or Korean characters. A model for writing process is proposed to explain the above difference, which takes…
Recent reasoning Large Language Models (LLMs) demonstrate remarkable problem-solving abilities but often generate long thinking traces whose utility is unclear. Our work aims to improve their efficiency, enabling them to reach high…
The translation of written language has been known since the 3rd century BC; however, its necessity has become increasingly common in the information age. Today, many translators exist, based on encoder-decoder deep architectures,…
Entropic uncertainty relations are interesting in their own rights as well as for a lot of applications. Keeping this in mind, we try to make the corresponding inequalities as tight as possible. The use of parametrized entropies also allows…
Generalizing the quantifiers used to classify correlations in bipartite systems, we define genuine total, quantum, and classical correlations in multipartite systems. The measure we give is based on the use of relative entropy to quantify…
Given a large real symmetric, positive semidefinite m-by-m matrix, the goal of this paper is to show how a numerical approximation of the entropy, given by the sum of the entropies of the individual eigenvalues, can be computed in an…
In a language corpus, the probability that a word occurs $n$ times is often proportional to $1/n^2$. Assigning rank, $s$, to words according to their abundance, $\log s$ vs $\log n$ typically has a slope of minus one. That simple Zipf's law…
Information theory has been taken as a prospective tool for quantifying the complexity of complex networks. In this paper, we first study the information entropy or uncertainty of a path using the information theory. Then we apply the path…
A simple proof is given for the monotonicity of entropy and Fisher information associated to sums of i.i.d. random variables. The proof relies on a characterization of maximal correlation for partial sums due to Dembo, Kagan and Shepp.
A longstanding debate in semiotics centers on the relationship between linguistic signs and their corresponding semantics: is there an arbitrary relationship between a word form and its meaning, or does some systematic phenomenon pervade?…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
This work studies the relation between two graph parameters, $\rho$ and $\Lambda$. For an undirected graph $G$, $\rho(G)$ is the growth rate of its universal covering tree, while $\Lambda(G)$ is a weighted geometric average of the vertex…
Defining the entropy of classical particles raises a number of paradoxes and ambiguities, some of which have been known for over a century. Several, such as Gibbs' paradox, involve the fact that classical particles are distinguishable, and…
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…