相关论文: Convergence and Error Bounds for Universal Predict…
TThe problem is to identify a probability associated with a set of natural numbers, given an infinite data sequence of elements from the set. If the given sequence is drawn i.i.d. and the probability mass function involved (the target)…
We introduce the notion of consistent error bound functions which provides a unifying framework for error bounds for multiple convex sets. This framework goes beyond the classical Lipschitzian and H\"olderian error bounds and includes…
Conformal prediction is a general distribution-free approach for constructing prediction sets combined with any machine learning algorithm that achieve valid marginal or conditional coverage in finite samples. Ordinal classification is…
In this paper, we study the almost sure boundedness and the convergence of the stochastic approximation (SA) algorithm. At present, most available convergence proofs are based on the ODE method, and the almost sure boundedness of the…
Many practical problems need the output of a machine learning model to satisfy a set of constraints, $K$. Nevertheless, there is no known guarantee that classical neural network architectures can exactly encode constraints while…
Previously referred to as `miraculous' in the scientific literature because of its powerful properties and its wide application as optimal solution to the problem of induction/inference, (approximations to) Algorithmic Probability (AP) and…
Suppose that we are given an infinite binary sequence which is random for a Bernoulli measure of parameter $p$. By the law of large numbers, the frequency of zeros in the sequence tends to~$p$, and thus we can get better and better…
We present and empirically evaluate an efficient algorithm that learns to aggregate the predictions of an ensemble of binary classifiers. The algorithm uses the structure of the ensemble predictions on unlabeled data to yield significant…
The authors present evidence for universality in numerical computations with random data. Given a (possibly stochastic) numerical algorithm with random input data, the time (or number of iterations) to convergence (within a given tolerance)…
The classical Poisson theorem says that if $\xi_1,\xi_2,...$ are i.i.d. 0--1 Bernoulli random variables taking on 1 with probability $p_n\equiv \la/n$ then the sum $S_n=\sum_{i=1}^n\xi_i$ is asymptotically in $n$ Poisson distributed with…
A consequence of de Finetti's representation theorem is that for every infinite sequence of exchangeable 0-1 random variables $(X_k)_{k\geq1}$, there exists a probability measure $\mu$ on the Borel sets of $[0,1]$ such that $\bar X_n =…
The problem is sequence prediction in the following setting. A sequence $x_1,...,x_n,...$ of discrete-valued observations is generated according to some unknown probabilistic law (measure) $\mu$. After observing each outcome, it is required…
Max-margin methods for binary classification such as the support vector machine (SVM) have been extended to the structured prediction setting under the name of max-margin Markov networks ($M^3N$), or more generally structural SVMs.…
This text presents an unified approach of probability and statistics in the pursuit of understanding and computation of randomness in engineering or physical or social system with prediction with generalizability. Starting from elementary…
The coding theorem for Kolmogorov complexity states that any string sampled from a computable distribution has a description length close to its information content. A coding theorem for resource-bounded Kolmogorov complexity is the key to…
We consider self-averaging sequences in which each term is a weighted average over previous terms. For several sequences of this kind it is known that they do not converge to a limit. These sequences share the property that $n$th term is…
Estimating a large alphabet probability distribution from a limited number of samples is a fundamental problem in machine learning and statistics. A variety of estimation schemes have been proposed over the years, mostly inspired by the…
Structured prediction can be considered as a generalization of many standard supervised learning tasks, and is usually thought as a simultaneous prediction of multiple labels. One standard approach is to maximize a score function on the…
We consider a setup in which Alice selects a pdf $f$ from a set of prescribed pdfs $\mathscr{P}$ and sends a prefix-free codeword $W$ to Bob in order to allow him to generate a single instance of the random variable $X\sim f$. We describe a…
We show how universal codes can be used for solving some of the most important statistical problems for time series. By definition, a universal code (or a universal lossless data compressor) can compress any sequence generated by a…