Related papers: Regression and Algorithmic Information Theory

The main question is: why and how can we ever predict based on a finite sample? The question is not answered by statistical learning theory. Here, I suggest that prediction requires belief in "predictability" of the underlying dependence,…

A key functionality of emerging connected autonomous systems such as smart cities, smart transportation systems, and the industrial Internet-of-Things, is the ability to process and learn from data collected at different physical locations.…

The incompressibility method is a counting argument in the framework of algorithmic complexity that permits discovering properties that are satisfied by most objects of a class. This paper gives a preliminary insight into Kolmogorov's…

In this paper we study subsequences of random numbers. In Kamae (1973), selection functions that depend only on coordinates are studied, and their necessary and sufficient condition for the selected sequences to be normal numbers is given.…

A drawback of Kolmogorov-Chaitin complexity (K) as a function from s to the shortest program producing s is its noncomputability which limits its range of applicability. Moreover, when strings are short, the dependence of K on a particular…

One of the most famous results in Complex Analysis is the Little Picard Theorem, that characterizes the image set of an arbitrary entire function. Specifically, the theorem states that this image set is either the whole complex plane or the…

Multitask learning, i.e. taking advantage of the relatedness of individual tasks in order to improve performance on all of them, is a core challenge in the field of machine learning. We focus on matrix regression tasks where the rank of the…

We show that the mutual information, in the sense of Kolmogorov complexity, of any pair of strings $x$ and $y$ is equal, up to logarithmic precision, to the length of the longest shared secret key that two parties, one having $x$ and the…

This paper proposes a fast and accurate method for sparse regression in the presence of missing data. The underlying statistical model encapsulates the low-dimensional structure of the incomplete data matrix and the sparsity of the…

We introduce a criterion, resilience, which allows properties of a dataset (such as its mean or best low rank approximation) to be robustly computed, even in the presence of a large fraction of arbitrary additional data. Resilience is a…

The main object of investigation in this paper is a very general regression model in optional setting - when an observed process is an optional semimartingale depending on an unknown parameter. It is well-known that statistical data may…

We extend the work of Narasimhan and Bilmes [30] for minimizing set functions representable as a difference between submodular functions. Similar to [30], our new algorithms are guaranteed to monotonically reduce the objective function at…

The classical coding theorem in Kolmogorov complexity states that if an $n$-bit string $x$ is sampled with probability $\delta$ by an algorithm with prefix-free domain then K$(x) \leq \log(1/\delta) + O(1)$. In a recent work, Lu and…

This paper establishes an abstract Korovkin-type approximation theorem in general spaces, extending the framework of approximation theory to accommodate broader contexts. A critical result supporting this theorem is the proof that any…

In this paper, we study the trace regression when a matrix of parameters B* is estimated via the convex relaxation of a rank-regularized regression or via regularized non-convex optimization. It is known that these estimators satisfy…

The theory of recursive functions is related in a well-known way to the notion of *least fixed points*, by endowing a set of partial functions with an ordering in terms of their domain of definition. When terms in the pure lambda-calculus…

Network regression with additive node-level random effects can be problematic when the primary interest is estimating unconditional regression coefficients and some covariates are exactly or nearly in the vector space of node-level effects.…

A rank estimator in robust regression is a minimizer of a function which depends (in addition to other factors) on the ordering of residuals but not on their values. Here we focus on the optimization aspects of rank estimators. We…

Performative prediction is a framework accounting for the shift in the data distribution induced by the prediction of a model deployed in the real world. Ensuring rapid convergence to a stable solution where the data distribution remains…

Representation learning is a key technique in modern machine learning that enables models to identify meaningful patterns in complex data. However, different methods tend to extract distinct aspects of the data, and relying on a single…