相关论文: Randomness and lowness notions via open covers
Despite the tremendous empirical success of deep learning models to solve various learning tasks, our theoretical understanding of their generalization ability is very limited. Classical generalization bounds based on tools such as the VC…
We elaborate the notions of Martin-L\"of and Schnorr randomness for real numbers in terms of uniform distribution of sequences. We give a necessary condition for a real number to be Schnorr random expressed in terms of classical uniform…
Since human randomness production has been studied and widely used to assess executive functions (especially inhibition), many measures have been suggested to assess the degree to which a sequence is random-like. However, each of them…
With the emergence of Large Language Models (LLMs), new methods in Information Retrieval are available in which relevance is estimated directly through language understanding and reasoning, instead of embedding similarity. We argue that…
Algorithmic theories of randomness can be related to theories of probabilistic sequence prediction through the notion of a predictor, defined as a function which supplies lower bounds on initial-segment probabilities of infinite sequences.…
Large language models display remarkable capabilities in logical and mathematical reasoning, allowing them to solve complex tasks. Interestingly, these abilities emerge in networks trained on the simple task of next-token prediction. In…
Existing definitions of the relativizations of \NCOne, \L\ and \NL\ do not preserve the inclusions $\NCOne \subseteq \L$, $\NL\subseteq \ACOne$. We start by giving the first definitions that preserve them. Here for \L\ and \NL\ we define…
These notes survey and explore an emerging method, which we call the low-degree method, for predicting and understanding statistical-versus-computational tradeoffs in high-dimensional inference problems. In short, the method posits that a…
Several easy to understand and computationally tractable imprecise probability models, like the Pari-Mutuel model, are derived from a given probability measure P_0. In this paper we investigate a family of such models, called Nearly-Linear…
LLMs are statistical models of language learning through stochastic gradient descent with a next token prediction objective. Prompting a popular view among AI modelers: LLMs are just next token predictors. While LLMs are engineered using…
Random linear codes (RLCs) are well known to have nice combinatorial properties and near-optimal parameters in many different settings. However, getting explicit constructions matching the parameters of RLCs is challenging, and RLCs are…
Bennett's notion of depth is usually considered to describe the usefulness and internal organization of the information encoded into an object such as an infinite binary sequence. We consider a natural way to relativize the notion of depth…
Normal numbers were introduced by Borel and later proven to be a weak notion of algorithmic randomness. We introduce here a natural relativization of normality based on generalized number representation systems. We explore the concepts of…
The rapid advancement of large language model (LLM) technology has led to diverse applications, many of which inherently require randomness, such as stochastic decision-making, gaming, scheduling, AI agents, and cryptography-related tasks.…
We develop new techniques for proving lower bounds on the least singular value of random matrices with limited randomness. The matrices we consider have entries that are given by polynomials of a few underlying base random variables. This…
Coded recurrent neural networks with three levels of sparsity are introduced. The first level is related to the size of messages, much smaller than the number of available neurons. The second one is provided by a particular coding rule,…
Pseudorandmness plays an important role in number theory, complexity theory and cryptography. Our aim is to use models of arithmetic to explain pseudorandomness by randomness. To this end we construct a set of models $\cal M$, a common…
A randomized algorithm for computing a compressed representation of a given rank-structured matrix $A \in \mathbb{R}^{N\times N}$ is presented. The algorithm interacts with $A$ only through its action on vectors. Specifically, it draws two…
In a metric space, such as the real numbers with their standard metric, a set A is open if and only if no sequence with terms outside of A has a limit inside A. Moreover, a metric space is compact if and only if every sequence has a…
This article deals with localization probability in a network of randomly distributed communication nodes contained in a bounded domain. A fraction of the nodes denoted as L-nodes are assumed to have localization information while the rest…