Related papers: Small Spans in Scaled Dimension
Large Language Models (LLMs) have achieved remarkable success in various natural language processing tasks, including language modeling, understanding, and generation. However, the increased memory and computational costs associated with…
This paper presents a construction of a proper and stable labelled sample compression scheme of size $O(\VCD^2)$ for any finite concept class, where $\VCD$ denotes the Vapnik-Chervonenkis Dimension. The construction is based on a well-known…
Following the success of the so-called algebraic approach to the study of decision constraint satisfaction problems (CSPs), exact optimization of valued CSPs, and most recently promise CSPs, we propose an algebraic framework for valued…
Recent empirical studies show three phenomena with increasing size of language models: compute-optimal size scaling, emergent capabilities, and performance plateauing. We present a simple unified mathematical framework to explain all of…
We prove that a Schauder frame for any separable Banach space is shrinking if and only if it has an associated space with a shrinking basis, and that a Schauder frame for any separable Banach space is shrinking and boundedly complete if and…
Let $\mathbb{P}= \{P_1, \cdots, P_{k}\in \mathbb{R}[y]\}$ be a collection of polynomials with distinct degrees and zero constant terms. We proved that there exists $\epsilon=\epsilon(\mathbb{P})>0$ such that, for any compact set $E \subset…
The Lopsided Lov\'{a}sz Local Lemma (LLLL) is a powerful probabilistic principle which has been used in a variety of combinatorial constructions. While originally a general statement about probability spaces, it has recently been…
Large language models (LLMs) have achieved remarkable success across a wide spectrum of tasks; however, they still face limitations in scenarios that demand long-term planning and spatial reasoning. To facilitate this line of research, in…
Kaplan et al. and Hoffmann et al. developed influential scaling laws for the optimal model size as a function of the compute budget, but these laws yield substantially different predictions. We explain the discrepancy by reproducing the…
Sentence encoders map sentences to real valued vectors for use in downstream applications. To peek into these representations - e.g., to increase interpretability of their results - probing tasks have been designed which query them for…
Bulatov (2008) gave a dichotomy for the counting constraint satisfaction problem #CSP. A problem from #CSP is characterised by a constraint language, which is a fixed, finite set of relations over a finite domain D. An instance of the…
This paper investigates Large Language Models (LLMs) ability to assess the economic soundness and theoretical consistency of empirical findings in spatial econometrics. We created original and deliberately altered "counterfactual" summaries…
By analyzing the concept of contextuality (Bell-Kochen-Specker) in terms of pre-and-post-selection (PPS), it is possible to assign definite values to observables in a new and surprising way. Physical reasons are presented for restrictions…
State Space Models (SSMs) are inherently recurrent along the sequence dimension, yet depth-recurrence - reusing the same block repeatedly across layers, as recently applied in looped transformers - has not been explored in this model…
Co-design conditions for the design of a jumping-rule and a sampled-data control law for impulsive and impulsive switched systems subject to aperiodic sampled-data measurements are provided. Semi-infinite discrete-time Lyapunov-Metzler…
In this paper we develop a general theoretical tool for the establishment of the boundedness of notoriously difficult operators (such as potentials) on certain specific types of rearrangement-invariant function spaces from analogous…
The rankable and compressible sets have been studied for more than a quarter of a century, ever since Allender [1] and Goldberg and Sipser [6] introduced the formal study of polynomial-time ranking. Yet even after all that time, whether the…
The motivating question for this work is a long standing open problem, posed by Nisan (1991), regarding the relative powers of algebraic branching programs (ABPs) and formulas in the non-commutative setting. Even though the general question…
In this paper we address the decision problem for a fragment of set theory with restricted quantification which extends the language studied in [4] with pair related quantifiers and constructs, in view of possible applications in the field…
Multidimensional scaling is an important dimension reduction tool in statistics and machine learning. Yet few theoretical results characterizing its statistical performance exist, not to mention any in high dimensions. By considering a…