Related papers: Tractability from overparametrization: The example…
Considering the classification problem, we summarize the nonparallel support vector machines with the nonparallel hyperplanes to two types of frameworks. The first type constructs the hyperplanes separately. It solves a series of small…
Let $\Omega\subset\mathbb{R}^\nu$, $\nu\ge 2$, be a $C^{1,1}$ domain whose boundary $\partial\Omega$ is either compact or behaves suitably at infinity. For $p\in(1,\infty)$ and $\alpha>0$, define \[…
For probability measures on countable spaces we derive distributional limits for empirical entropic optimal transport quantities. More precisely, we show that the empirical optimal transport plan weakly converges to a centered Gaussian…
We improve the inequality used in Pronzato [2003. Removing non-optimal support points in D-optimum design algorithms. Statist. Probab. Lett. 63, 223-228] to remove points from the design space during the search for a $D$-optimum design. Let…
We establish a precise connection between two elliptic quasilinear problems with Dirichlet data in a bounded domain of $\mathbb{R}^{N}.$ The first one, of the form \[ -\Delta_{p}u=\beta(u)| \nabla u| ^{p}+\lambda f(x)+\alpha, \] involves a…
In this paper, we study the predict-then-optimize problem where the output of a machine learning prediction task is used as the input of some downstream optimization problem, say, the objective coefficient vector of a linear program. The…
Statistical mechanics is used to study unrealizable generalization in two large feed-forward neural networks with binary weights and output, a perceptron and a tree committee machine. The student is trained by a teacher being larger, i.e.…
We study information theoretic limits of recovering an unknown $n$ dimensional, complex signal vector $\mathbf{x}_\star$ with unit norm from $m$ magnitude-only measurements of the form $y_i = |(\mathbf{A} \mathbf{x}_\star)_i|^2, \; i = 1,2…
Statistical-physics calculations in machine learning and theoretical neuroscience often involve lengthy derivations that obscure physical interpretation. Here, we give concise, non-replica derivations of several key results and highlight…
We show that the maximal determinant D(n) for $n \times n$ ${\pm 1}$-matrices satisfies $R(n) := D(n)/n^{n/2} \ge \kappa_d > 0$. Here $n^{n/2}$ is the Hadamard upper bound, and $\kappa_d$ depends only on $d := n-h$, where $h$ is the maximal…
In solving a system of $n$ linear equations in $d$ variables $Ax=b$, the condition number of the $n,d$ matrix $A$ measures how much errors in the data $b$ affect the solution $x$. Estimates of this type are important in many inverse…
The estimation law of unknown parameters vector ${\theta}$ is proposed for one class of nonlinearly parametrized regression equations $y\left( t \right) = \Omega \left( t \right)\Theta \left( \theta \right)$. We restrict our attention to…
Current deep neural networks are highly overparameterized (up to billions of connection weights) and nonlinear. Yet they can fit data almost perfectly through variants of gradient descent algorithms and achieve unexpected levels of…
Understanding the character of electronic excitations is important in computational and reaction mechanistic studies, but their classification from simulations remains an open problem. Distances based on optimal transport have proven very…
The Support Vector Machine (SVM) is one of the most widely used classification methods. In this paper, we consider the soft-margin SVM used on data points with independent features, where the sample size $n$ and the feature dimension $p$…
Over-parametrization is an important technique in training neural networks. In both theory and practice, training a larger network allows the optimization algorithm to avoid bad local optimal solutions. In this paper we study a closely…
Minimizing cross-entropy over the softmax scores of a linear map composed with a high-capacity encoder is arguably the most popular choice for training neural networks on supervised learning tasks. However, recent works show that one can…
Data vectors generalise finite multisets: they are finitely supported functions into a commutative monoid. We study the question if a given data vector can be expressed as a finite sum of others, only assuming that 1) the domain is…
In this note we give a new effective proof method for the equivalence of the notions of testability and nondeterministic testability for uniform hypergraph parameters. We provide the first effective upper bound on the sample complexity of…
In high-dimensional classification settings, we wish to seek a balance between high power and ensuring control over a desired loss function. In many settings, the points most likely to be misclassified are those who lie near the decision…