Related papers: Coincidence Points of Parameterized Generalized Eq…
In this article, we generalize some frequently used metrical notions such as: completeness, closedness, continuity, g-continuity and compatibility to relation-theoretic setting and utilize these relatively weaker notions to prove results on…
Machine learning algorithms have been used widely in various applications and areas. To fit a machine learning model into different problems, its hyper-parameters must be tuned. Selecting the best hyper-parameter configuration for machine…
The bias-variance trade-off is a central concept in supervised learning. In classical statistics, increasing the complexity of a model (e.g., number of parameters) reduces bias but also increases variance. Until recently, it was commonly…
We study the problem of parameter-free stochastic optimization, inquiring whether, and under what conditions, do fully parameter-free methods exist: these are methods that achieve convergence rates competitive with optimally tuned methods,…
Solutions to network optimization problems have greatly benefited from developments in nonlinear analysis, and, in particular, from developments in convex optimization. A key concept that has made convex and nonconvex analysis an important…
Recent successes of massively overparameterized models have inspired a new line of work investigating the underlying conditions that enable overparameterized models to generalize well. This paper considers a framework where the possibly…
Tuning parameters are parameters involved in an estimating procedure for the purpose of reducing the risk of some other estimator. Examples include the degree of penalization in penalized regression and likelihood problems, as well as the…
Typically, the sequence of points generated by an optimization algorithm may have multiple limit points. Under convexity assumptions, however, (sub)gradient methods are known to generate a convergent sequence of points. In this paper, we…
This paper revisits the parametric analysis of semidefinite optimization problems with respect to the perturbation of the objective function along a fixed direction. We review the notions of invariancy set, nonlinearity interval, and…
In this paper we propose a general algorithmic framework for first-order methods in optimization in a broad sense, including minimization problems, saddle-point problems and variational inequalities. This framework allows to obtain many…
In the finite dimensional case, mean-type mappings, their invariant means, relations between the uniqueness of invariant means and convergence of orbits of the mapping, are considered. In particular it is shown, that the uniqueness of an…
Let us assume that $f$ is a continuous function defined on the unit ball of $\mathbb R^d$, of the form $f(x) = g (A x)$, where $A$ is a $k \times d$ matrix and $g$ is a function of $k$ variables for $k \ll d$. We are given a budget $m \in…
An algorithm capable of finding a likely global optimum (minimum) and a set of sub-optimal points for arbitrary generic functions of several variables is presented. The algorithm is designed to deal even with functions of complex behavior,…
We study problem-dependent rates, i.e., generalization errors that scale near-optimally with the variance, the effective loss, or the gradient norms evaluated at the "best hypothesis." We introduce a principled framework dubbed "uniform…
This paper derives new algorithms for signomial programming, a generalization of geometric programming. The algorithms are based on a generic principle for optimization called the MM algorithm. In this setting, one can apply the…
In this note, we consider a framework for the analysis of iterative algorithms which can described in terms of a structured set-valued operator. More precisely, at each point in the ambient space, we assume that the value of operator can be…
Ultrafunctions are a particular class of generalized functions defined on a hyperreal field $\mathbb{R}^{*}\supset\mathbb{R}$ that allow to solve variational problems with no classical solutions. We recall the construction of ultrafunctions…
The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used. One avenue of inquiry has been to look at these loss functions in terms of their properties as scoring rules…
We provide a new perspective on the study of parameterized optimization problems. Our approach combines methods for post-optimal sensitivity analysis and ordinary differential equations to quantify the uncertainty in the minimizer due to…
The analysis of data sometimes requires fitting many free parameters in a theory to a large number of data points. Questions naturally arise about the compatibility of specific subsets of the data, such as those from a particular experiment…