Related papers: A physical study of the LLL algorithm

The purpose of this paper is to extend the result of arXiv:1810.00823 to mixed H\"older functions on $[0,1]^d$ for all $d \ge 1$. In particular, we prove that by sampling an $\alpha$-mixed H\"older function $f : [0,1]^d \rightarrow…

We show numerically that correlation length at the critical point in the five-dimensional Ising model varies with system size L as L^{5/4}, rather than proportional to L as in standard finite size scaling (FSS) theory. Our results confirm a…

The stochastic sandpile model (SSM) is a generalisation of the standard Abelian sandpile model (ASM), in which topplings of unstable vertices are made random. When unstable, a vertex sends one grain to each of its neighbours independently…

The success of today's large language models (LLMs) depends on the observation that larger models perform better. However, the origin of this neural scaling law, that loss decreases as a power law with model size, remains unclear. We…

Large language models (LLMs) are increasingly used as decision-support tools in data-constrained scientific workflows, where correctness and validity are critical. However, evaluation practices often emphasize stability or reproducibility…

Supervised fine-tuning (SFT) is crucial for aligning Large Language Models (LLMs) with human instructions. The primary goal during SFT is to select a small yet representative subset of training data from the larger pool, such that…

We introduce a generalization of the continued fraction and Chakravala algorithms for solving the Pell equation, utilizing the LLL-algorithm for rank 2 lattices.

Directed sandpile models with different toppling rules are studied by means of numerical simulations in two dimensions, with the purpose of determining the different universality classes. It is concluded that the random-threshold directed…

When training deep neural networks, a model's generalization error is often observed to follow a power scaling law dependent both on the model size and the data size. Perhaps the best known example of such scaling laws are for…

When data is plentiful, the loss achieved by well-trained neural networks scales as a power-law $L \propto N^{-\alpha}$ in the number of network parameters $N$. This empirical scaling law holds for a wide variety of data modalities, and may…

Laplace approximations are commonly used to approximate high-dimensional integrals in statistical applications, but the quality of such approximations as the dimension of the integral grows is not well understood. In this paper, we prove a…

Prior work has shown that large language models (LLMs) often converge to accurate input embedding for numbers, based on sinusoidal representations. In this work, we quantify that these representations are in fact strikingly systematic, to…

General Relativity simplifies dramatically in the limit that the number of spacetime dimensions D is infinite: it reduces to a theory of non-interacting particles, of finite radius but vanishingly small cross sections, which do not emit nor…

In recent work by L. Levine and Y. Peres, it was observed that three models for particle aggregation on the lattice - the divisible sandpile, rotor-router aggregation, and internal diffusion limited aggregation - share a common scaling…

We consider $d$-dimensional linear stochastic approximation algorithms (LSAs) with a constant step-size and the so called Polyak-Ruppert (PR) averaging of iterates. LSAs are widely applied in machine learning and reinforcement learning…

The Superficial Alignment Hypothesis posits that almost all of a language model's abilities and knowledge are learned during pre-training, while post-training is about giving a model the right style and format. We re-examine these claims by…

Locality-sensitive hashing (LSH) is an important tool for managing high-dimensional noisy or uncertain data, for example in connection with data cleaning (similarity join) and noise-robust search (similarity search). However, for a number…

Although demonstrating remarkable performance on reasoning tasks, Large Language Models (LLMs) still tend to fabricate unreliable responses when confronted with problems that are unsolvable or beyond their capability, severely undermining…

Theory and application of stochastic approximation (SA) have become increasingly relevant due in part to applications in optimization and reinforcement learning. This paper takes a new look at SA with constant step-size $\alpha>0$, defined…

Decision boundary, the subspace of inputs where a machine learning model assigns equal classification probabilities to two classes, is pivotal in revealing core model properties and interpreting behaviors. While analyzing the decision…