Related papers: Majorizing Measures for the Optimizer
Optimization, a key tool in machine learning and statistics, relies on regularization to reduce overfitting. Traditional regularization methods control a norm of the solution to ensure its smoothness. Recently, topological methods have…
We consider a class of convex optimization problems over the simplex of probability measures. Our framework comprises optimal experimental design (OED) problems, in which the measure over the design space indicates which experiments are…
We develop a novel procedure for estimating the optimizer of general convex stochastic optimization problems of the form $\min_{x\in\mathcal{X}} \mathbb{E}[F(x,\xi)]$, when the given data is a finite independent sample selected according to…
Approximating integrals is a fundamental task in probability theory and statistical inference, and their applied fields of signal processing, and Bayesian learning, as soon as expectations over probability distributions must be computed…
A wide variety of problems in machine learning, including exemplar clustering, document summarization, and sensor placement, can be cast as constrained submodular maximization problems. Unfortunately, the resulting submodular optimization…
Talagrand conjectured that if a family of sets $\mathcal{F}$ over $X = \{ 1,2,\cdots, N \}$ is of large measure, then constant times of unions of sets in $\mathcal{F}$ will cover a large portion of the power set of $X$. This conjecture is a…
Computer science theory provides many different measures of complexity of a system including Kolmogorov complexity, logical depth, computational depth, and Levin complexity. However, these measures are all defined only for deterministic…
Calibration has emerged as a foundational goal in ``trustworthy machine learning'', in part because of its strong decision theoretic semantics. Independent of the underlying distribution, and independent of the decision maker's utility…
An algorithm for computing the Karcher mean of $n$ positive definite matrices is proposed, based on the majorization-minimization (MM) principle. The proposed MM algorithm is parameter-free, does not need to choose step sizes, and has a…
Ergodic optimization is the study of problems relating to maximizing orbits, maximizing invariant measures and maximum ergodic averages. An orbit of a dynamical system is called f-maximizing if the time average of the real-valued function f…
Machine learning algorithms frequently require careful tuning of model hyperparameters, regularization terms, and optimization parameters. Unfortunately, this tuning is often a "black art" that requires expert experience, unwritten rules of…
We propose and study a novel collection of signed measures, which will be apply called Taylor measures. Stochastic versions of the new measures are also defined and studied. We illustrate, through examples, how the deterministic and…
A wide array of machine learning problems are formulated as the minimization of the expectation of a convex loss function on some parameter space. Since the probability distribution of the data of interest is usually unknown, it is is often…
In this paper, we generalize the chance optimization problems and introduce constrained volume optimization where enables us to obtain convex formulation for challenging problems in systems and control. We show that many different problems…
Modern applications require methods that are computationally feasible on large datasets but also preserve statistical efficiency. Frequently, these two concerns are seen as contradictory: approximation methods that enable computation are…
The superiorization methodology (SM) is an optimization heuristic in which an iterative algorithm, which aims to solve a particular problem, is ``superiorized'' to promote solutions that are improved with respect to some secondary…
Effective regularisation of neural networks is essential to combat overfitting due to the large number of parameters involved. We present an empirical analogue to the Lipschitz constant of a feed-forward neural network, which we refer to as…
The paper proposes a novel regularization procedure for machine learning. The proposed high-order regularization (HR) provides new insight into regularization, which is widely used to train a neural network that can be utilized to…
In this paper, we solve a maximization problem where the objective function is quadratic and convex or concave and the constraints set is the reachable value set of a convergent discrete-time affine system. Moreover, we assume that the…
First-order optimization methods have attracted a lot of attention due to their practical success in many applications, including in machine learning. Obtaining convergence guarantees and worst-case performance certificates for first-order…