Related papers: On Empirical Risk Minimization with Dependent and …

Empirical risk minimization (ERM) can be computationally expensive, with standard solvers scaling poorly even in the convex setting. We propose a novel lossless compression framework for convex ERM based on color refinement, extending prior…

We establish risk bounds for Regularized Empirical Risk Minimizers (RERM) when the loss is Lipschitz and convex and the regularization function is a norm. In a first part, we obtain these results in the i.i.d. setup under subgaussian…

In this paper, a novel approach to the problem of estimating the heavy-tail exponent alpha>0 of a distribution is proposed. It is based on the fact that block-maxima of size m of the independent and identically distributed data scale at a…

The solution to empirical risk minimization with $f$-divergence regularization (ERM-$f$DR) is presented under mild conditions on $f$. Under such conditions, the optimal measure is shown to be unique. Examples of the solution for particular…

We study square loss in a realizable time-series framework with martingale difference noise. Our main result is a fast rate excess risk bound which shows that whenever a trajectory hypercontractivity condition holds, the risk of the…

The generalization error (risk) of a supervised statistical learning algorithm quantifies its prediction ability on previously unseen data. Inspired by exponential tilting, \citet{li2020tilted} proposed the {\it tilted empirical risk} (TER)…

In clinical trials, studies often present longitudinal data or clustered data. These studies are commonly analyzed using linear mixed models (LMMs), usually considering Gaussian assumptions for random effect and error terms. Recently,…

We study the problem of estimating Dynamic Discrete Choice (DDC) models, also known as offline Maximum Entropy-Regularized Inverse Reinforcement Learning (offline MaxEnt-IRL) in machine learning. The objective is to recover reward or $Q^*$…

The relationship between a response variable and its covariates can vary significantly, especially in scenarios where covariates take on extremely high or low values. This paper introduces a max-linear tail regression model specifically…

As an effective nonparametric method, empirical likelihood (EL) is appealing in combining estimating equations flexibly and adaptively for incorporating data information. To select important variables and estimating equations in the sparse…

The contribution of this paper is to introduce change of measure based techniques for the rare-event analysis of heavy-tailed stochastic processes. Our changes-of-measure are parameterized by a family of distributions admitting a mixture…

Empirical risk minimization (ERM) is a cornerstone of modern machine learning (ML), supported by advances in optimization theory that ensure efficient solutions with provable algorithmic and statistical learning rates. Privacy, memory,…

Over the past decades, there has been a surge of interest in studying low-dimensional structures within high-dimensional data. Statistical factor models $-$ i.e., low-rank plus diagonal covariance structures $-$ offer a powerful framework…

Minimizing the empirical risk is a popular training strategy, but for learning tasks where the data may be noisy or heavy-tailed, one may require many observations in order to generalize well. To achieve better performance under less…

The non-asymptotic tail bounds of random variables play crucial roles in probability, statistics, and machine learning. Despite much success in developing upper bounds on tail probability in literature, the lower bounds on tail…

We introduce Invariant Risk Minimization (IRM), a learning paradigm to estimate invariant correlations across multiple training distributions. To achieve this goal, IRM learns a data representation such that the optimal classifier, on top…

In the framework of censored data modeling, the classical linear regression model that assumes normally distributed random errors has received increasing attention in recent years, mainly for mathematical and computational convenience.…

Tensor regression is an important tool for tensor data analysis, but existing works have not considered the impact of outliers, making them potentially sensitive to such data points. This paper proposes a low tubal rank robust regression…

We propose an $\ell_1$-penalized estimator for high-dimensional models of Expected Shortfall (ES). The estimator is obtained as the solution to a least-squares problem for an auxiliary dependent variable, which is defined as a…

In this study, we propose a robust mixture regression procedure based on the skew t distribution to model heavy-tailed and/or skewed errors in a mixture regression setting. Using the scale mixture representation of the skew t distribution,…