Related papers: Universal Learning Theory
Quantum theory was discovered in an adventurous way, under the urge to solve puzzles-like the spectrum of the blackbody radiation-that haunted the physics community at the beginning of the 20th century. It soon became clear, though, that…
This Inaugural Lecture was given at Royal Holloway University of London in 1996. It covers an introduction to machine learning and describes various theoretical advances and practical projects in the field. The Lecture here is presented in…
Deep learning is usually described as an experiment-driven field under continuous criticizes of lacking theoretical foundations. This problem has been partially fixed by a large volume of literature which has so far not been well organized.…
Humans can learn a variety of concepts and skills incrementally over the course of their lives while exhibiting many desirable properties, such as continual learning without forgetting, forward transfer and backward transfer of knowledge,…
Einstein's general relativity is the best available theory of gravity. In recent years, spectacular proofs of Einstein's theory have been conducted, which have aroused interest that goes far beyond the narrow circle of specialists. The aim…
Deep learning relies on a very specific kind of neural networks: those superposing several neural layers. In the last few years, deep learning achieved major breakthroughs in many tasks such as image analysis, speech recognition, natural…
This paper shows how universal learning can be achieved with expert advice. To this aim, we specify an experts algorithm with the following characteristics: (a) it uses only feedback from the actions actually chosen (bandit setup), (b) it…
The Euclidean algorithm makes possible a simple but powerful generalization of Taylor's theorem. Instead of expanding a function in a series around a single point, one spreads out the spectrum to include any number of points with given…
We study generation through the lens of statistical learning theory. First, we abstract and formalize the results of Gold [1967], Angluin [1979], Angluin [1980] and Kleinberg and Mullainathan [2024] in terms of a binary hypothesis class…
Traditional machine learning mainly supervised learning, follows the assumptions of closed-world learning, i.e., for each testing class, a training class is available. However, such machine learning models fail to identify the classes which…
We prove a universality theorem for learning with random features. Our result shows that, in terms of training and generalization errors, a random feature model with a nonlinear activation function is asymptotically equivalent to a…
We are at the cusp of a transition from "learning from data" to "learning what data to learn from" as a central focus of artificial intelligence (AI) research. While the first-order learning problem is not completely solved, large models…
In 1895, Cantor showed that between every two countable dense real sets, there is an order isomorphism. In fact, there is always such an order isomorphism, which is the restriction of a universal entire function.
We consider the fundamental question of learnability of a hypotheses class in the supervised learning setting and in the general learning setting introduced by Vladimir Vapnik. We survey classic results characterizing learnability in term…
Time-series data exists in every corner of real-world systems and services, ranging from satellites in the sky to wearable devices on human bodies. Learning representations by extracting and inferring valuable information from these time…
This paper presents an overview of some of the concepts of Bayesian Learning. The number of scientific and industrial applications of Bayesian learning has been growing in size rapidly over the last few decades. This process has started…
We present a universally consistent learning rule whose expected error is monotone non-increasing with the sample size under every data distribution. The question of existence of such rules was brought up in 1996 by Devroye, Gy\"orfi and…
This small overview of the General Theory is the kind of summary I would have liked to have read, before embarking in a comprehensive study of the General Theory at the time I was a student. As shown here, the main ideas are quite simple…
The universal approximation theorem asserts that a single hidden layer neural network approximates continuous functions with any desired precision on compact sets. As an existential result, the universal approximation theorem supports the…
In this short note, we point out a mistake in G.Cybenko's proof of his version of the universal approximation theorem which has been widely cited. This mistake might not be easily fixable along the idea of his proof and it also leads to an…