Related papers: Criticality in Formal Languages and Statistical Ph…
Massive language models are the core of modern NLP modeling and have been shown to encode impressive amounts of commonsense and factual information. However, that knowledge exists only within the latent parameters of the model, inaccessible…
We derive a well-defined renormalized version of mutual information that allows to estimate the dependence between continuous random variables in the important case when one is deterministically dependent on the other. This is the situation…
A longstanding debate in semiotics centers on the relationship between linguistic signs and their corresponding semantics: is there an arbitrary relationship between a word form and its meaning, or does some systematic phenomenon pervade?…
Traditional linguistic theories have largely regard language as a formal system composed of rigid rules. However, their failures in processing real language, the recent successes in statistical natural language processing, and the findings…
Natural language semantics has recently sought to combine the complementary strengths of formal and distributional approaches to meaning. More specifically, proposals have been put forward to augment formal semantic machinery with…
We address this work to investigate some statistical properties of symbolic sequences generated by a numerical procedure in which the symbols are repeated following a power law probability density. In this analysis, we consider that the sum…
Mutual information is fundamentally important for measuring statistical dependence between variables and for quantifying information transfer by signaling and communication mechanisms. It can, however, be challenging to evaluate for…
As a core element of culture, images transform perception into structured representations and undergo evolution similar to natural languages. Given that visual input accounts for 60% of human sensory experience, it is natural to ask whether…
Recently, a model of opinion formation with kinetic exchanges has been proposed in which a spontaneous symmetry breaking transition was reported [M. Lallouache et al, Phys. Rev. E, {\bf 82} 056112 (2010)]. We generalise the model to…
Lexical Semantic Change is the study of how the meaning of words evolves through time. Another related question is whether and how lexical relations over pairs of words, such as synonymy, change over time. There are currently two competing,…
Despite the wide usage of information as a concept in science, we have yet to develop a clear & concise scientific definition. This paper is aimed at laying the foundations for a new theory concerning the mechanics of information alongside…
It is now well established empirically that financial price changes are distributed according to a power law, with cubic exponent. This is a fascinating regularity, as it holds for various classes of securities, on various markets, and on…
Processes involving bursts of activity separated by quiescent periods occur across diverse systems and scales. In human dynamics, these phenomena have been described by power-law inter-event time distributions, $P(t)\sim t^{-\alpha}$, with…
Quantitative linguistics has provided us with a number of empirical laws that characterise the evolution of languages and competition amongst them. In terms of language usage, one of the most influential results is Zipf's law of word…
We present a theoretical and empirical investigation of the statistical behaviour of the words in a text produced by human language. To this aim, we analyse the word distribution of various texts of Italian language selected from a specific…
In a previous paper [1] we considered the question, "What underlying property of nature is responsible for the second law?" A simple answer can be stated in terms of information: The fundamental loss of information gives rise to the second…
The importance of transformations and normal forms in logic programming, and generally in computer science, is well documented. This paper investigates transformations and normal forms in the context of Defeasible Logic, a simple but…
Power laws arise in a variety of phenomena ranging from matter undergoing phase transition to the distribution of word frequencies in the English language. Usually, their presence is only apparent when data is abundant, and accurately…
We show that size-rank distributions with power-law decay (often only over a limited extent) observed in a vast number of instances in a widespread family of systems obey Tsallis statistics. The theoretical framework for these distributions…
Human languages have evolved to be structured through repeated language learning and use. These processes introduce biases that operate during language acquisition and shape linguistic systems toward communicative efficiency. In this paper,…