Related papers: Linear Models are Most Favorable among Generalized…

This book is meant to provide an introduction to linear models and the theories behind them. Our goal is to give a rigorous introduction to the readers with prior exposure to ordinary least squares. In machine learning, the output is…

We consider the problem of predicting as well as the best linear combination of d given functions in least squares regression, and variants of this problem including constraints on the parameters of the linear combination. When the input…

This paper studies a class of exponential family models whose canonical parameters are specified as linear functionals of an unknown infinite-dimensional slope function. The optimal minimax rates of convergence for slope function estimation…

We consider a linear model where the coefficients - intercept and slopes - are random with a law in a nonparametric class and independent from the regressors. Identification often requires the regressors to have a support which is the whole…

We introduce the class of rational plane curves parameterizable by conics as an extension of the family of curves parameterizable by lines (also known as monoid curves). We show that they are the image of monoid curves via suitable…

Most systems and learning algorithms optimize average performance or average loss -- one reason being computational complexity. However, many objectives of practical interest are more complex than simply average loss. This arises, for…

Designing learning algorithms that are resistant to perturbations of the underlying data distribution is a problem of wide practical and theoretical importance. We present a general approach to this problem focusing on unsupervised…

The problem of low-rank matrix estimation recently received a lot of attention due to challenging applications. A lot of work has been done on rank-penalized methods and convex relaxation, both on the theoretical and applied sides. However,…

Theory of graphical models has matured over more than three decades to provide the backbone for several classes of models that are used in a myriad of applications such as genetic mapping of diseases, credit risk evaluation, reliability and…

We provide a general constrained risk inequality that applies to arbitrary non-decreasing losses, extending a result of Brown and Low [Ann. Stat. 1996]. Given two distributions $P_0$ and $P_1$, we find a lower bound for the risk of…

We study the implicit bias of generic optimization methods, such as mirror descent, natural gradient descent, and steepest descent with respect to different potentials and norms, when optimizing underdetermined linear regression or…

In this paper it is shown that the compact linearization approach, that has been previously proposed only for binary quadratic problems with assignment constraints, can be generalized to arbitrary linear equations with positive coefficients…

We show that the average stability notion introduced by \cite{kearns1999algorithmic, bousquet2002stability} is invariant to data preconditioning, for a wide class of generalized linear models that includes most of the known exp-concave…

Missing values arise in most real-world data sets due to the aggregation of multiple sources and intrinsically missing information (sensor failure, unanswered questions in surveys...). In fact, the very nature of missing values usually…

In this paper, we are concerned with the generalization performance of non-parametric estimation for pairwise learning. Most of the existing work requires the hypothesis space to be convex or a VC-class, and the loss to be convex. However,…

The convex envelopes of the direct discrete measures, for the sparsity of vectors or for the low-rankness of matrices, have been utilized extensively as practical penalties in order to compute a globally optimal solution of the…

We compute a variance lower bound for unbiased estimators in specified statistical models. The construction of the bound is related to the original Cram\'er-Rao bound, although it does not require the differentiability of the model.…

We consider the minimum distance projection in the $L_2$-norm from an arbitrary point in an $n$-dimensional, Euclidian space onto the canonical simplex. It is shown that this problem reduces to a univariate problem that can be solved by a…

Linear codes have been an interesting topic in both theory and practice for many years. In this paper, a class of $q$-ary linear codes with few weights are presented and their weight distributions are determined using Gauss periods. Some of…

Learning of low-rank matrices is fundamental to many machine learning applications. A state-of-the-art algorithm is the rank-one matrix pursuit (R1MP). However, it can only be used in matrix completion problems with the square loss. In this…