Related papers: The Density of Linear-time Properties
We propose a new finetuning method to provide pre-trained large language models (LMs) the ability to scale test-time compute through the diffusion framework. By increasing the number of diffusion steps, we show our finetuned models achieve…
In this paper we study the density in the real line of oscillating sequences of the form $$ (g(k)\cdot F(k\alpha))_{k \in \mathbb{N}} ,$$ where $g$ is a positive increasing function and $F$ a real continuous 1-periodic function. This…
We consider a type of long-range percolation problem on the positive integers, motivated by earlier work of others on the appearance of (in)finite words within a site percolation model. The main issue is whether a given infinite binary word…
In this paper we consider a problem of searching a space of predictive models for a given training data set. We propose an iterative procedure for deriving a sequence of improving models and a corresponding sequence of sets of non-linear…
Recent research has extensively studied how large language models manipulate integers in specific arithmetic tasks, and on a more fundamental level, how they represent numeric values. These previous works have found that language model…
We revisit the complexity of building, given a two-dimensional string of size $n$, an indexing structure that allows locating all $k$ occurrences of a two-dimensional pattern of size $m$. While a structure of size $\mathcal{O}(n)$ with…
This paper investigates the influence of environmental noise on the characteristic timescale of the dynamics of density-dependent populations. General results are obtained on the statistics of time spent in rarity and time spent in…
Modern machine learning models with a large number of parameters often generalize well despite perfectly interpolating noisy training data - a phenomenon known as benign overfitting. A foundational explanation for this in linear…
We present upper and lower bounds for the prediction error of the Lasso. For the case of random Gaussian design, we show that under mild conditions the prediction error of the Lasso is up to smaller order terms dominated by the prediction…
We introduce the concept of a class of graphs, or more generally, relational structures, being locally tree-decomposable. There are numerous examples of locally tree-decomposable classes, among them the class of planar graphs and all…
We show that every isoperimetric set in R^N with density is bounded if the density is continuous and bounded by above and below. This improves the previously known boundedness results, which basically needed a Lipschitz assumption; on the…
Many geometric optimization problems can be reduced to finding points in space (centers) minimizing an objective function which continuously depends on the distances from the centers to given input points. Examples are $k$-Means, Geometric…
Motivated by the recent detection of an enhanced clustering signal along the major axis of haloes in N-body simulations, we derive a formula for the anisotropic density distribution around haloes and voids on large scales. Our model, which…
Masked language modeling is a widely used method for learning language representations, where the model predicts a randomly masked word in each input. However, this approach typically considers only a single correct answer during training,…
The Lasso is a popular model selection and estimation procedure for linear models that enjoys nice theoretical properties. In this paper, we study the Lasso estimator for fitting autoregressive time series models. We adopt a double…
In a recent letter [Europhys. Lett. 95, 13001 (2011)] the question of whether the density of a time-dependent quantum system determines its external potential was reformulated as a fixed point problem. This idea was used to generalize the…
Arguably, omega-regular languages play an important role as a specification formalism in many approaches to systems monitoring via runtime verification. However, since their elements are infinite words, not every omega-regular language can…
We consider a class of continuous-time stochastic growth models on $d$-dimensional lattice with non-negative real numbers as possible values per site. The class contains examples such as binary contact path process and potlatch process. We…
Recently, Large Language Models (LLMs) have achieved remarkable success. A key factor behind this success is the scaling law observed by OpenAI. Specifically, for models with Transformer architecture, the test loss exhibits a power-law…
The Lasso is one of the most important approaches for parameter estimation and variable selection in high dimensional linear regression. At the heart of its success is the attractive rate of convergence result even when $p$, the dimension…