Related papers: Approximability of deep computations
Coherent lower previsions are general probabilistic models allowing incompletely specified probability distributions. However, for complete description of a coherent lower prevision -- even on finite underlying sample spaces -- an infinite…
We initiate a formal study of reproducibility in optimization. We define a quantitative measure of reproducibility of optimization procedures in the face of noisy or error-prone operations such as inexact or stochastic gradient computations…
This review is an extended version of the Seoul ICM 2014 proceedings.It is a short overview of the "topological recursion", a relation appearing in the asymptotic expansion of many integrable systems and in enumerative problems. We recall…
The ultimate limits of computation are not just logical, but physical. We investigate the physical resources -- time, energy, entropy, and free energy -- required to perform computational work. We apply the resulting measures of physical…
We present a method for computing the topological entropy of one-dimensional maps. As an approximation scheme, the algorithm converges rapidly and provides both upper and lower bounds.
In this manuscript, we study the learning of deep attention neural networks, defined as the composition of multiple self-attention layers, with tied and low-rank weights. We first establish a mapping of such models to sequence multi-index…
In this article, we consider a simple representation for real numbers and propose top-down procedures to approximate various algebraic and transcendental operations with arbitrary precision. Detailed algorithms and proofs are provided to…
Semigroup theory is a branch of abstract algebra, and it provides mathematical tools for the theory of computation. Finite semigroups can describe state transition systems and thus they model physically realizable computers. Engineering…
This paper is concerned with the study of a family of fixed point iterations combining relaxation with different inertial (acceleration) principles. We provide a systematic, unified and insightful analysis of the hypotheses that ensure…
Implicit computational complexity, which aims at characterizing complexity classes by machine-independent means, has traditionally been based, on the one hand, on programs and deductive formalisms for free algebras, and on the other hand on…
Superposition -- when a neural network represents more ``features'' than it has dimensions -- seems to pose a serious challenge to mechanistically interpreting current AI systems. Existing theory work studies \emph{representational}…
Nonlinear models and optimization methods have successfully tackled a rapidly growing set of problems in recent years. Indeed, a relatively small toolbox of such models and methods can provide sufficient performance across a large landscape…
We investigate the computational properties of basic mathematical notions pertaining to $\mathbb{R}\rightarrow \mathbb{R}$-functions and subsets of $\mathbb{R}$, like finiteness, countability, (absolute) continuity, bounded variation,…
The classical simulation of physical processes using standard models of computation is fraught with problems. On the other hand, attempts at modelling real-world computation with the aim of isolating its hypercomputational content have…
We study approximation of embeddings between finite dimensional L_p spaces in the quantum model of computation. For the quantum query complexity of this problem matching (up to logarithmic factors) upper and lower bounds are obtained. The…
We study the computational problem of rigorously describing the asymptotic behaviour of topological dynamical systems up to a finite but arbitrarily small pre-specified error. More precisely, we consider the limit set of a typical orbit,…
A deep approximation is an approximating function defined by composing more than one layer of simple functions. We study deep approximations of functions of one variable using layers consisting of low-degree polynomials or simple conformal…
A major part of computability theory focuses on the analysis of a few structures of central importance. As a tool, the method of coding with first-order formulas has been applied with great success. For instance, in the c.e. Turing degrees,…
Deep learning models have been the subject of study from various perspectives, for example, their training process, interpretation, generalization error, robustness to adversarial attacks, etc. A trained model is defined by its decision…
We present a theory for simultaneous approximation of the score function and its derivatives, enabling the handling of data distributions with low-dimensional structure and unbounded support. Our approximation error bounds match those in…