Related papers: Two-Dimensional Pattern Languages
In this paper we introduce some families of fuzzy L-systems and investigate their properties. We further discuss the relationship between fuzzy L languages and the fuzzy languages generated by fuzzy grammar proposed in Ref.[3,5]. A measure…
Lexical semantic typology has identified important cross-linguistic generalizations about the variation and commonalities in polysemy patterns---how languages package up meanings into words. Recent computational research has enabled…
We establish a relation between fully extended $2$-dimensional TQFTs and recognisable weighted formal languages, rational biprefix codes and lattice TFTs. We show the equivalence of $2D$ closed TFTs and rational exchangeable series and we…
Language models have primarily been evaluated with perplexity. While perplexity quantifies the most comprehensible prediction performance, it does not provide qualitative information on the success or failure of models. Another approach for…
We provide the results of pattern recognition experiments on mathematical expressions. We give a few examples of conjectured results. None of which was thoroughly checked for novelty. We did not attempt to prove all the relations found and…
Crosslingual transfer is crucial to contemporary language models' multilingual capabilities, but how it occurs is not well understood. We ask what happens to a monolingual language model when it begins to be trained on a second language.…
Pre-trained word embeddings are widely used for transfer learning in natural language processing. The embeddings are continuous and distributed representations of the words that preserve their similarities in compact Euclidean spaces.…
Psychological research consistently finds that human ratings of words across diverse semantic scales can be reduced to a low-dimensional form with relatively little information loss. We find that the semantic associations encoded in the…
Large Language Models (LLMs) exhibit impressive performance on a range of NLP tasks, due to the general-purpose linguistic knowledge acquired during pretraining. Existing model interpretability research (Tenney et al., 2019) suggests that a…
Many complex generative systems use languages to create structured objects. We consider a model of random languages, defined by weighted context-free grammars. As the distribution of grammar weights broadens, a transition is found from a…
Language models are typically evaluated on their success at predicting the distribution of specific words in specific contexts. Yet linguistic knowledge also encodes relationships between contexts, allowing inferences between word…
The downward closure of a word language is the set of all (not necessarily contiguous) subwords of its members. It is well-known that the downward closure of any language is regular. While the downward closure appears to be a powerful…
This article is an introduction to formal languages from the point of view of combinatorial group theory. Group theoretic applications are included and language classes are defined algebraically.
Probing techniques for large language models (LLMs) have primarily focused on English, overlooking the vast majority of the world's languages. In this paper, we extend these probing methods to a multilingual context, investigating the…
Almost none of the 2,000+ languages spoken in Africa have widely available automatic speech recognition systems, and the required data is also only available for a few languages. We have experimented with two techniques which may provide…
A systematic way of defining variants of a modeling language is useful for adopting the language to domain or project specific needs. Variants can be obtained by adopting the syntax or semantics of the language. In this paper, we take a…
Language models based on the Transformer architecture achieve excellent results in many language-related tasks, such as text classification or sentiment analysis. However, despite the architecture of these models being well-defined, little…
We investigate a family of rule-based logics. The focus is on very expressive languages. We provide a range of characterization results for the expressive powers of the logics and relate them with corresponding game systems.
We find generating functions for the number of words avoiding certain patterns or sets of patterns on at most 2 distinct letters and determine which of them are equally avoided. We also find the exact number of words avoiding certain…
This paper proposes a mechanism for learning pattern correspondences between two languages from a corpus of translated sentence pairs. The proposed mechanism uses analogical reasoning between two translations. Given a pair of translations,…