Related papers: Double descent in the condition number
Anomaly detection is a branch of data analysis and machine learning which aims at identifying observations that exhibit abnormal behaviour. Be it measurement errors, disease development, severe weather, production quality default(s) (items)…
In an underdetermined system of equations $Ax=y$, where $A$ is an $m\times n$ matrix, only $u$ of the entries of $y$ with $u < m$ are known. Thus $E_jw$, called `measurements', are known for certain $j\in J \subset \{0,1,\ldots,m-1\}$ where…
In this paper, we propose second-order sufficient optimality conditions for a very general nonconvex constrained optimization problem, which covers many prominent mathematical programs.Unlike the existing results in the literature, our…
Extensive empirical evidence reveals that, for a wide range of different learning methods and datasets, the risk curve exhibits a double-descent (DD) trend as a function of the model size. In a recent paper [Zeyu,Kammoun,Thrampoulidis,2019]…
A general sufficient condition for the convergence of subsequences of solutions of non-autonomous, nonlinear difference equations and systems is obtained. For higher order equations the delay sizes and patterns play essential roles in…
In most social choice settings, the participating agents express their preferences over the different alternatives in the form of linear orderings. While this clearly simplifies preference elicitation, it inevitably leads to poor…
Decision-making problems are commonly formulated as optimization problems, which are then solved to make optimal decisions. In this work, we consider the inverse problem where we use prior decision data to uncover the underlying…
The inverse problem methodology is a commonly-used framework in the sciences for parameter estimation and inference. It is typically performed by fitting a mathematical model to noisy experimental data. There are two significant sources of…
Motivated by recently emerging problems in machine learning and statistics, we propose data models which relax the familiar i.i.d. assumption. In essence, we seek to understand what it means for data to come from a set of probability…
We study estimation and detection of high-order moment and cumulant tensors from $n$ i.i.d.\ observations of a $p$-dimensional random vector, with performance measured in tensor spectral norm. Under sub-Gaussianity, we show that the minimax…
Let $n>m$, and let $A$ be an $(m\times n)$-matrix of full rank. Then obviously the estimate $\|Ax\|\leq\|A\|\|x\|$ holds for the euclidean norm of $x$ and $Ax$ and the spectral norm as the assigned matrix norm. We study the sets of all $x$…
Out-of-distribution (OOD) detection is essential for ensuring the reliability and safety of machine learning systems. In recent years, it has received increasing attention, particularly through post-hoc detection and training-based methods.…
The mathematical modeling of numerous real-world applications results in hierarchical optimization problems with two decision makers where at least one of them has to solve an optimal control problem of ordinary or partial differential…
Understanding generalization and estimation error of estimators for simple models such as linear and generalized linear models has attracted a lot of attention recently. This is in part due to an interesting observation made in machine…
Recently, several authors have suggested the use of first order methods, such as fast dual ascent and the alternating direction method of multipliers, for embedded model predictive control. The main reason is that they can be implemented…
Two-sample inference for the difference of population means typically relies upon a Central Limit Theorem approximation. When data are drawn from a Negative Binomial distribution, previous work of Shilane et al. (2010) showed that a Normal…
It has been observed by Belkin et al.\ that over-parametrized neural networks exhibit a `double descent' phenomenon. That is, as the model complexity (as reflected in the number of features) increases, the test error initially decreases,…
One of the central puzzles in modern machine learning is the ability of heavily overparametrized models to generalize well. Although the low-dimensional structure of typical datasets is key to this behavior, most theoretical studies of…
In deep learning, it is common to use more network parameters than training points. In such scenarioof over-parameterization, there are usually multiple networks that achieve zero training error so that thetraining algorithm induces an…
The exploration of complex physical or technological processes usually requires exploiting available information from different sources: (i) physical laws often represented as a family of parameter dependent partial differential equations…