Related papers: Regression and Algorithmic Information Theory
In analogy of classical Kolmogorov complexity we develop a theory of the algorithmic information in bits contained in any one of continuously many pure quantum states: quantum Kolmogorov complexity. Classical Kolmogorov complexity coincides…
Consider a nonparametric regression model with one-sided errors and regression function in a general H\"older class. We estimate the regression function via minimization of the local integral of a polynomial approximation. We show uniform…
Suppose the nonparametric regression function of a response variable $Y$ on covariates $X$ and $Z$ is an affine function of $X$ such that the slope $\beta$ and the intercept $\alpha$ are real valued measurable functions on the range of the…
Solomonoff's general theory of inference and the Minimum Description Length principle formalize Occam's razor, and hold that a good model of data is a model that is good at losslessly compressing the data, including the cost of describing…
We introduce a statistical physics inspired supervised machine learning algorithm for classification and regression problems. The method is based on the invariances or stability of predicted results when known data is represented as…
This paper presents a theory of non-linear integer/real arithmetic and algorithms for reasoning about this theory. The theory can be conceived as an extension of linear integer/real arithmetic with a weakly-axiomatized multiplication…
We consider the regression problem where the dependence of the response Y on a set of predictors X is fully captured by the regression function E(Y | X)=g(B'X), for an unknown function g and low rank parameter B matrix. We combine neural…
We propose regression networks for the problem of few-shot classification, where a classifier must generalize to new classes not seen in the training set, given only a small number of examples of each class. In high dimensional embedding…
Classical versions of Kolmogorov complexity are incomputable. Nevertheless, in 1975 Solovay showed that there are computable functions $f > K+O(1)$ such that for infinitely many strings $\sigma$, $f(\sigma)=K(\sigma)+O(1)$, where $K$…
Given two jointly distributed random variables $(X,Y)$, a functional representation of $X$ is a random variable $Z$ independent of $Y$, and a deterministic function $g(\cdot, \cdot)$ such that $X=g(Y,Z)$. The problem of finding a minimum…
We consider the problem of robust polynomial regression, where one receives samples $(x_i, y_i)$ that are usually within $\sigma$ of a polynomial $y = p(x)$, but have a $\rho$ chance of being arbitrary adversarial outliers. Previously, it…
Developing new ways to estimate probabilities can be valuable for science, statistics, and engineering. By considering the information content of different output patterns, recent work invoking algorithmic information theory has shown that…
Although procedural generation is popular among game developers, academic research on the topic has primarily focused on new applications, with some research into empirical analysis. In this paper we relate theoretical work in information…
Logistic regression models are a popular and effective method to predict the probability of categorical response data. However inference for these models can become computationally prohibitive for large datasets. Here we adapt ideas from…
We consider classification and regression tasks where we have missing data and assume that the (clean) data resides in a low rank subspace. Finding a hidden subspace is known to be computationally hard. Nevertheless, using a non-proper…
Recently, there was a substantial progress in the problem of sampling recovery on function classes with mixed smoothness. Mostly, it has been done by proving new and sometimes optimal upper bounds for both linear sampling recovery and for…
Reduced Rank Regression (RRR) is a widely used method for multi-response regression. However, RRR assumes a linear relationship between features and responses. While linear models are useful and often provide a good approximation, many…
In domains like bioinformatics, information retrieval and social network analysis, one can find learning tasks where the goal consists of inferring a ranking of objects, conditioned on a particular target object. We present a general kernel…
Subsampling is an efficient method to deal with massive data. In this paper, we investigate the optimal subsampling for linear quantile regression when the covariates are functions. The asymptotic distribution of the subsampling estimator…
In this paper, we construct an estimator of an errors-in-variables linear regression model. The regression model leads to a constrained total least squares problems with row and column constraints. Although this problem can be numerically…