Related papers: Mathematical foundations of matrix syntax

Accurate syntactic representations are essential for robust generalization in natural language. Recent work has found that pre-training can teach language models to rely on hierarchical syntactic features - as opposed to incorrect linear…

A possible explanation for the impressive performance of masked language model (MLM) pre-training is that such models have learned to represent the syntactic structures prevalent in classical NLP pipelines. In this paper, we propose a…

Formal logic has often been seen as uniquely placed to analyze mathematical argumentation. While formal logic is certainly necessary for a complete understanding of mathematical practice, it is not sufficient. Important aspects of…

Action models are semantic structures similar to Kripke models that represent a change in knowledge in an epistemic setting. Whereas the language of action model logic embeds the semantic structure of an action model directly within the…

The foundations of mathematics have long been considered settled by the Zermelo-Fraenkel-Choice axioms. But set theory abounds in models with different truths and even classical questions such as the measurability of projective sets can…

Traditional linguistic theories have largely regard language as a formal system composed of rigid rules. However, their failures in processing real language, the recent successes in statistical natural language processing, and the findings…

A major problem in the study of large language models is to understand their inherent low-dimensional structure. We introduce an approach to study the low-dimensional structure of language models at a model-agnostic level: as sequential…

We present a general class of machine learning algorithms called parametric matrix models. In contrast with most existing machine learning models that imitate the biology of neurons, parametric matrix models use matrix equations that…

Prompted models have demonstrated impressive few-shot learning abilities. Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. These compositions are…

Large language models (LLMs) process and predict sequences containing text to answer questions, and address tasks including document summarization, providing recommendations, writing software and solving quantitative problems. We provide a…

Patterns are fundamental to human cognition, enabling the recognition of structure and regularity across diverse domains. In this work, we focus on structural repeats, patterns that arise from the repetition of hierarchical relations within…

Logical relations constitute a key method for reasoning about contextual equivalence of programs in higher-order languages. They are usually developed on a per-case basis, with a new theory required for each variation of the language or of…

Analysing several characteristic mathematical models: natural and real numbers, Euclidean geometry, group theory, and set theory, I argue that a mathematical model in its final form is a junction of a set of axioms and an internal partial…

Linear algebra algorithms often require some sort of iteration or recursion as is illustrated by standard algorithms for Gaussian elimination, matrix inversion, and transitive closure. A key characteristic shared by these algorithms is that…

In recent years, promising mathematical models have been suggested which aim to describe conscious experience and its relation to the physical domain. Whereas the axioms and metaphysical ideas of these theories have been carefully…

Language models (LMs) are said to be exhibiting reasoning, but what does this entail? We assess definitions of reasoning and how key papers in the field of natural language processing (NLP) use the notion and argue that the definitions…

This paper describes a computational framework for a grammar architecture in which different linguistic domains such as morphology, syntax, and semantics are treated not as separate components but compositional domains. Word and phrase…

This article is an exploratory account of the the non-monotonic behaviour of conceptual associations in the light of context. Computational approximations of conceptual space are furnished by semantic space models which are emerging from…

A network representation is useful for describing the structure of a large variety of complex systems. However, most real and engineered systems have multiple subsystems and layers of connectivity, and the data produced by such systems is…

If language evolved by sexual selection to display superior intelligence, then we require conversational skills, to impress other people, gain high social status, and get a mate. Conversational skills include a Theory of Mind, a sense of…