Related papers: The Semantics of Rank Polymorphism
The evolving capabilities of large language models are accompanied by growing sizes and deployment costs, necessitating effective inference optimisation techniques. We propose a novel pruning method utilising centrality measures from graph…
Relative monads provide a controlled view of computation. We generalise the monadic metalanguage to a relative setting and give a complete semantics with strong relative monads. Adopting this perspective, we generalise two existing program…
Multi-level languages and Arrows both facilitate metaprogramming, the act of writing a program which generates a program. The arr function required of all Arrows turns arbitrary host language expressions into guest language expressions;…
The notion of `stable rank' of a matrix is central to the analysis of randomized matrix algorithms, covariance estimation, deep neural networks, and recommender systems. We compare the properties of the stable rank and intrinsic dimension…
Low-rank adaptation (LoRA) has been prominently employed for parameter-efficient fine-tuning of large language models (LLMs). However, the limited expressive capacity of LoRA, stemming from the low-rank constraint, has been recognized as a…
We investigate the class of visibly pushdown languages in the sliding window model. A sliding window algorithm for a language $L$ receives a stream of symbols and has to decide at each time step whether the suffix of length $n$ belongs to…
The ability of Large Language Models (LLMs) to solve complex tasks has made them crucial in the development of AI-based applications. However, the high computational requirements to fine-tune these LLMs on downstream tasks pose significant…
One of the most attractive features of untyped languages is the flexibility in term creation and manipulation. However, with such power comes the responsibility of ensuring the correctness of these operations. A solution is adding run-time…
I present the most fundamental features of an implemented system designed to manipulate representations of regular languages. The system is structured into two layers, allowing regular languages to be represented in an increasingly compact,…
Reasoning is a fundamentally algorithmic task. Yet current work on LLM-based reasoning relies on free-form generation whose theoretical guarantees (soundness, completeness, complexity, optimality) remain poorly understood. We argue that we…
The paper is devoted to showing how to systematically design a programming language in 'reverse order', i.e. from denotations to syntax. This construction is developed in an algebraic framework consisting of three many-sorted algebras: of…
Traditionally, natural language processing (NLP) models often use a rich set of features created by linguistic expertise, such as semantic representations. However, in the era of large language models (LLMs), more and more tasks are turned…
In this paper we consider the class of lambda-nondeterministic linear automata as a model of the class of linear languages. As usual in other automata models, lambda-moves do not increase the acceptance power. The main contribution of this…
English verbs have multiple forms. For instance, talk may also appear as talks, talked or talking, depending on the context. The NLP task of lemmatization seeks to map these diverse forms back to a canonical one, known as the lemma. We…
Low Rank Adaptation (LoRA) is a popular Parameter Efficient Fine Tuning (PEFT) method that effectively adapts large pre-trained models for downstream tasks. LoRA parameterizes model updates using low-rank matrices at each layer,…
Large Language Models (LLMs) have transformed natural language processing and extended their powerful capabilities to multi-modal domains. As LLMs continue to advance, it is crucial to develop diverse and appropriate metrics for their…
Runtime efficiency and termination are crucial properties in the studies of program verification. Instead of dealing with these issues in an ad hoc manner, it would be useful to develop a robust framework in which such properties are…
Probabilistic programming languages rely fundamentally on some notion of sampling, and this is doubly true for probabilistic programming languages which perform Bayesian inference using Monte Carlo techniques. Verifying samplers - proving…
In previous work we describe a novel approach to dependent typing based on a multivalued term language. In this technical report we formalise the runtime, a kind of operational semantics, for that language. We describe a fairly…
Statistical studies of languages have focused on the rank-frequency distribution of words. Instead, we introduce here a measure of how word ranks change in time and call this distribution \emph{rank diversity}. We calculate this diversity…