Related papers: A physical study of the LLL algorithm

Fine-tuning Large Language Models (LLMs) on specific datasets is a common practice to improve performance on target tasks. However, this performance gain often leads to overfitting, where the model becomes too specialized in either the task…

In the rapidly growing literature on explanation algorithms, it often remains unclear what precisely these algorithms are for and how they should be used. In this position paper, we argue for a novel and pragmatic perspective: Explainable…

The difficulties arising in the investigation of finite-size scaling in $d$--dimensional O(n) systems with strong anisotropy and/or long-range interaction, decaying with the interparticle distance $r$ as $r^{-d-\sigma}$ ($0<\sigma\leq2$),…

Statistical query (SQ) algorithms are algorithms that have access to an {\em SQ oracle} for the input distribution $D$ instead of i.i.d.~ samples from $D$. Given a query function $\phi:X \rightarrow [-1,1]$, the oracle returns an estimate…

We define a new version of sandpile model which is very similar to Abelian Sandpile Model (ASM), but the height variables are continuous ones. With the toppling rule we define in our model, we show that the model can be mapped to ASM, so…

Recent studies have shown that heavy tails can emerge in stochastic optimization and that the heaviness of the tails have links to the generalization error. While these studies have shed light on interesting aspects of the generalization…

In Large Language Model (LLM) development, Reinforcement Learning from Human Feedback (RLHF) is crucial for aligning models with human values and preferences. RLHF traditionally relies on the Kullback-Leibler (KL) divergence between the…

DeepLLL algorithm (Schnorr, 1994) is a famous variant of LLL lattice basis reduction algorithm, and PotLLL algorithm (Fontein et al., 2014) and $S^2$LLL algorithm (Yasuda and Yamaguchi, 2019) are recent polynomial-time variants of DeepLLL…

Large language models (LLMs) are deployed on increasingly complex tasks that require multi-step decision-making. Understanding their algorithmic reasoning abilities is therefore crucial. However, we lack a diagnostic benchmark for…

It has been experimentally observed in recent years that multi-layer artificial neural networks have a surprising ability to generalize, even when trained with far more parameters than observations. Is there a theoretical basis for this?…

The height probabilities for the recurrent configurations in the Abelian Sandpile Model on the square lattice have analytic expressions, in terms of multidimensional quadratures. At first, these quantities have been evaluated numerically…

The past few years have seen impressive progress in the development of deep generative models capable of producing high-dimensional, complex, and photo-realistic data. However, current methods for evaluating such models remain incomplete:…

In this paper, we prove the large deviation principle (LDP) for stochastic differential equations driven by stochastic integrals in one dimension. The result can be proved with a minimal use of rough path theory, and this implies the LDP…

This paper proposes a model, the linear model, for randomly generating logic programs with low density of rules and investigates statistical properties of such random logic programs. It is mathematically shown that the average number of…

We introduce an exactly solvable lattice model that reveals a universal finite-size scaling law for configurational entropy driven purely by geometry. Using exact enumeration via Burnside's lemma, we compute the entropy for diverse 1D, 2D,…

A data analyst might worry about generalization if dropping a very small fraction of data points from a study could change its substantive conclusions. Checking this non-robustness directly poses a combinatorial optimization problem and is…

The credit on {\it reduction theory} goes back to the work of Lagrange, Gauss, Hermite, Korkin, Zolotarev, and Minkowski. Modern reduction theory is voluminous and includes the work of A. Lenstra, H. Lenstra and L. Lovasz who created the…

We study the transfer learning (TL) for the functional linear regression (FLR) under the Reproducing Kernel Hilbert Space (RKHS) framework, observing that the TL techniques in existing high-dimensional linear regression are not compatible…

This paper proposes a specific type of Local Linear Model, the Shuffled Linear Model (SLM), that can be used as a universal approximator. Local operating points are chosen randomly and linear models are used to approximate a function or…

Density Functional Theory calculations traditionally suffer from an inherent cubic scaling with respect to the size of the system, making big calculations extremely expensive. This cubic scaling can be avoided by the use of so-called linear…