Related papers: The Generalized Universal Law of Generalization
Comparing probability distributions is an indispensable and ubiquitous task in machine learning and statistics. The most common way to compare a pair of Borel probability measures is to compute a metric between them, and by far the most…
The paper gives a bound on the generalization error of the Gibbs algorithm, which recovers known data-independent bounds for the high temperature range and extends to the low-temperature range, where generalization depends critically on the…
Belief function theory provides a flexible way to combine information provided by different sources. This combination is usually followed by a decision making which can be handled by a range of decision rules. Some rules help to choose the…
We generalize Huberman-Rudnick universal scaling law for all periodic windows of the logistic map and show the robustness of $q$-Gaussian probability distributions in the vicinity of chaos threshold. Our scaling relation is universal for…
Deep neural networks are often seen as different from other model classes by defying conventional notions of generalization. Popular examples of anomalous generalization behaviour include benign overfitting, double descent, and the success…
Over-parameterized models can perfectly learn various types of data distributions, however, generalization error is usually lower for real data in comparison to artificial data. This suggests that the properties of data distributions have…
We consider an agent interacting with an unknown environment. The environment is a function which maps natural numbers to natural numbers; the agent's set of hypotheses about the environment contains all such functions which are computable…
Terms of Church's $\lambda$-calculus can be considered equivalent along many different definitions, but context equivalence is certainly the most direct and universally accepted one. If the underlying calculus becomes probabilistic,…
The Levenshtein distance is an important tool for the comparison of symbolic sequences, with many appearances in genome research, linguistics and other areas. For efficient applications, an approximation by a distance of smaller…
We establish a strong law of large numbers for one-dimensional continuous-time random walks in dynamic random environments under two main assumptions: the environment is required to satisfy a decoupling inequality that can be interpreted as…
The uncertainty principle can be understood as constraining the probability of winning a game in which Alice measures one of two conjugate observables, such as position or momentum, on a system provided by Bob, and he is to guess the…
Nourdin et al. [9] established the following universality result: if a sequence of off-diagonal homogeneous polynomial forms in i.i.d. standard normal random variables converges in distribution to a normal, then the convergence also holds…
In a language corpus, the probability that a word occurs $n$ times is often proportional to $1/n^2$. Assigning rank, $s$, to words according to their abundance, $\log s$ vs $\log n$ typically has a slope of minus one. That simple Zipf's law…
A quantum probability model is introduced and used to explain human probability judgment errors including the conjunction, disjunction, inverse, and conditional fallacies, as well as unpacking effects and partitioning effects. Quantum…
We consider the sum of power weighted nearest neighbor distances in a sample of size n from a multivariate density f of possibly unbounded support. We give various criteria guaranteeing that this sum satisfies a law of large numbers for…
The generalization error of a learning algorithm refers to the discrepancy between the loss of a learning algorithm on training data and that on unseen testing data. Various information-theoretic bounds on the generalization error have been…
Diagonalization in the spirit of Cantor's diagonal arguments is a widely used tool in theoretical computer sciences to obtain structural results about computational problems and complexity classes by indirect proofs. The Uniform…
We study a generalized version of Wyner's common information problem (also coined the distributed source simulation problem). The original common information problem consists in understanding the minimum rate of the common input to…
Proximal distance algorithms combine the classical penalty method of constrained minimization with distance majorization. If $f(\boldsymbol{x})$ is the loss function, and $C$ is the constraint set in a constrained minimization problem, then…
Generalizing empirical findings to new environments, settings, or populations is essential in most scientific explorations. This article treats a particular problem of generalizability, called "transportability", defined as a license to…