Related papers: A Language and Its Dimensions: Intrinsic Dimension…
Some results of a computational experiment for determining the intrinsic dimension of linguistic varieties for the Bengali and Russian languages are presented. At the same time, both sets of words and sets of bigrams in these languages were…
Language exhibits a fractal structure in its information-theoretic complexity (i.e. bits per token), with self-similarity across scales and long-range dependence (LRD). In this work, we investigate whether large language models (LLMs) can…
The main goal of this paper has a double purpose. On the one hand, we propose a new definition in order to compute the fractal dimension of a subset respect to any fractal structure, which completes the theory of classical box-counting…
Word embeddings represent language vocabularies as clouds of $d$-dimensional points. We investigate how information is conveyed by the general shape of these clouds, instead of representing the semantic meaning of each token. Specifically,…
In this paper, we present Russian language datasets in the digital humanities domain for the evaluation of word embedding techniques or similar language modeling and feature learning algorithms. The datasets are split into two task types,…
This paper provides a new model to compute the fractal dimension of a subset on a generalized-fractal space. Recall that fractal structures are a perfect place where a new definition of fractal dimension can be given, so we perform a…
Most modern NLP systems make use of pre-trained contextual representations that attain astonishingly high performance on a variety of tasks. Such high performance should not be possible unless some form of linguistic structure inheres in…
Dense word embeddings, which encode semantic meanings of words to low dimensional vector spaces have become very popular in natural language processing (NLP) research due to their state-of-the-art performances in many NLP tasks. Word…
The aim of this work is to explore the possible limitations of existing methods of cross-language word embeddings evaluation, addressing the lack of correlation between intrinsic and extrinsic cross-language evaluation methods. To prove…
We study the fractal structure of language, aiming to provide a precise formalism for quantifying properties that may have been previously suspected but not formally shown. We establish that language is: (1) self-similar, exhibiting…
By virtue of linguistic compositionality, few syntactic rules and a finite lexicon can generate an unbounded number of sentences. That is, language, though seemingly high-dimensional, can be explained using relatively few degrees of…
Inflection graphs are highly complex networks representing relationships between inflectional forms of words in human languages. For so-called synthetic languages, such as Latin or Polish, they have particularly interesting structure due to…
In this paper we analyse the fractal structure of long human-language records by mapping large samples of texts onto time series. The particular mapping set up in this work is inspired on linguistic basis in the sense that is retains {\em…
A theory of resource-bounded dimension is developed using gales, which are natural generalizations of martingales. When the resource bound \Delta (a parameter of the theory) is unrestricted, the resulting dimension is precisely the…
Most of the known methods for estimating the fractal dimension of fractal sets are based on the evaluation of a single geometric characteristic, e.g. the volume of its parallel sets. We propose a method involving the evaluation of several…
Rapidly increasing quality of AI-generated content makes it difficult to distinguish between human and AI-generated texts, which may lead to undesirable consequences for society. Therefore, it becomes increasingly important to study the…
A representation of frequency of strings of length K in complete genomes of many organisms in a square has led to seemingly self-similar patterns when K increases. These patterns are caused by under-represented strings with a certain…
Perfect fractals are mathematical objects that, because they are generated by recursive processes, have self-similarity and infinite complexity. In particular, they also have a fractional dimension. Although several proposals for the study…
Multilingual LLMs demonstrate strong performance across diverse languages, yet there has been limited systematic analysis of how language information is structured within their internal representation space and how it emerges across layers.…
Generating coherent, grammatically correct, and meaningful text is very challenging, however, it is crucial to many modern NLP systems. So far, research has mostly focused on English language, for other languages both standardized datasets,…