Related papers: Sharpening Occam's Razor
We propose a new polynomial-time algorithm for linear programming. We further extend the ideas used in this new linear programming algorithm for nonlinear programming problems. The new algorithm is based on the idea of treating the…
In sparse recovery we are given a matrix $A$ (the dictionary) and a vector of the form $A X$ where $X$ is sparse, and the goal is to recover $X$. This is a central notion in signal processing, statistics and machine learning. But in…
How do we compare between hypotheses that are entirely consistent with observations? The marginal likelihood (aka Bayesian evidence), which represents the probability of generating our observations from a prior, provides a distinctive…
We strengthen H\"older's inequality. The new family of sharp inequalities we obtain might be thought of as an analog of Pythagorean theorem for the $L^p$ spaces. Our reasonings rely upon Bellman functions of four variables.
We give new algorithms based on the sum-of-squares method for tensor decomposition. Our results improve the best known running times from quasi-polynomial to polynomial for several problems, including decomposing random overcomplete…
Here we argue that the notion of falsifiability, a key concept in defining a valid scientific theory, can be quantified using Bayesian Model Selection, which is a standard tool in modern statistics. This relates falsifiability to the…
We obtain exact, simple and very compact expressions for the linearization coefficients of the products of orthogonal polynomials; both the conventional Clebsch-Gordan-type and the modified version. The expressions are general depending…
Linear regression is a basic and widely-used methodology in data analysis. It is known that some quantum algorithms efficiently perform least squares linear regression of an exponentially large data set. However, if we obtain values of the…
The present work explores the theoretical limits of Machine Learning (ML) within the framework of Kolmogorov's theory of Algorithmic Probability, which clarifies the notion of entropy as Expected Kolmogorov Complexity and formalizes other…
We construct a new family of strong linearizations of rational matrices considering the polynomial part of them expressed in a basis that satisfies a three term recurrence relation. For this purpose, we combine the theory developed by…
We discuss pedagogical problems associated with the conventional approach in teaching relativity and the potential value of "Lorentzian route to relativity" for addressing these pedagogical problems. We think that Lorentzian approach may be…
This paper describes a novel method to approximate the polynomial coefficients of regression functions, with particular interest on multi-dimensional classification. The derivation is simple, and offers a fast, robust classification…
The standard definition of PAC learning (Valiant 1984) requires learners to succeed under all distributions -- even ones that are intractable to sample from. This stands in contrast to samplable PAC learning (Blum, Furst, Kearns, and Lipton…
This paper deals with robust regression and subspace estimation and more precisely with the problem of minimizing a saturated loss function. In particular, we focus on computational complexity issues and show that an exact algorithm with…
Low rank tensor decompositions are a powerful tool for learning generative models, and uniqueness results give them a significant advantage over matrix decomposition methods. However, tensors pose significant algorithmic challenges and…
Deterministic global optimization of nonlinear models is important in many scientific and engineering applications. This framework typically involves repeatedly solving convex relaxations of the nonconvex problem, meaning that the strength…
This paper argues that DNNs implement a computational Occam's razor -- finding the `simplest' algorithm that fits the data -- and that this could explain their incredible and wide-ranging success over more traditional statistical methods.…
Given a way to evaluate an unknown polynomial with integer coefficients, we present new algorithms to recover its nonzero coefficients and corresponding exponents. As an application, we adapt this interpolation algorithm to the problem of…
The purpose of the paper is to introduce two new algorithms. The first one computes a linear recursion for proper hypergeometric multisums, by treating one summation variable at a time, and provides rational certificates along the way. A…
Hartmanis used Kolmogorov complexity to provide an alternate proof of the classical result of Baker, Gill, and Solovay that there is an oracle relative to which P is not NP. We refine the technique to strengthen the result, constructing an…