Related papers: A Short Proof of Strassen's Theorem Using Convex A…
This paper is concerned with the study of constrained statistical learning problems, the unconstrained version of which are at the core of virtually all of modern information processing. Accounting for constraints, however, is paramount to…
This article develops a duality principle for a class of optimization problems in $\mathbb{R}^n$. The results are obtained based on standard tools of convex analysis and on a well known result of Toland for D.C. optimization. Global…
This paper conducts sensitivity analysis of random constraint and variational systems related to stochastic optimization and variational inequalities. We establish efficient conditions for well-posedness, in the sense of robust Lipschitzian…
We discuss a general approach to handling "multiple hypotheses" testing in the case when a particular hypothesis states that the vector of parameters identifying the distribution of observations belongs to a convex compact set associated…
A new concise proof is given of a duality theorem connecting completely monotone relaxation functions with Bernstein class creep functions. The proof makes use of the theory of complete Bernstein functions and Stieltjes functions and is…
High-dimensional data have recently been analyzed because of data collection technology evolution. Although many methods have been developed to gain sparse recovery in the past two decades, most of these methods require selection of tuning…
Stochastic difference-of-convex (DC) optimization is prevalent in numerous machine learning applications, yet its convergence properties under small batch sizes remain poorly understood. Existing methods typically require large batches or…
General nonconvex optimization problems are studied by using the canonical duality-triality theory. The triality theory is proved for sums of exponentials and quartic polynomials, which solved an open problem left in 2003. This theory can…
Many large-scale constrained optimization problems can be formulated as bilevel distributed optimization tasks over undirected networks, where agents collaborate to minimize a global cost function while adhering to constraints, relying only…
Stochastic optimization problems often involve data distributions that change in reaction to the decision variables. This is the case for example when members of the population respond to a deployed classifier by manipulating their features…
This paper is aimed to prove the strong duality theorem for continuous-time linear programming problems in which the coefficients are assumed to be piecewise continuous functions. The previous paper proved the strong duality theorem for the…
Variational analysis provides the theoretical foundations and practical tools for constructing optimization algorithms without being restricted to smooth or convex problems. We survey the central concepts in the context of a concrete but…
The optimal control problem of stochastic systems is commonly solved via robust or scenario-based optimization methods, which are both challenging to scale to long optimization horizons. We cast the optimal control problem of a stochastic…
Dantzig and Eaves claimed that fundamental duality theorems of linear programming were a trivial consequence of Fourier elimination. Another property of Fourier elimination is considered here, regarding the existence of implicit equalities…
Stochastic contraction analysis is a recently developed tool for studying the global stability properties of nonlinear stochastic systems, based on a differential analysis of convergence in an appropriate metric. To date, stochastic…
Gradient dominance property is a condition weaker than strong convexity, yet sufficiently ensures global convergence even in non-convex optimization. This property finds wide applications in machine learning, reinforcement learning (RL),…
In recent years, a range of measures of partial stochastic dominance have been introduced. These measures attempt to determine the extent to which one distribution is dominated by another. We assess these measures from intuitive, axiomatic,…
The basic disentanglement theorem established by the present authors states that estimates on a weighted geometric mean over (convex) families of functions can be disentangled into quantitatively linked estimates on each family separately.…
Understanding the implicit bias of training algorithms is of crucial importance in order to explain the success of overparametrised neural networks. In this paper, we study the dynamics of stochastic gradient descent over diagonal linear…
A stochastic iterative algorithm approximating second-order information using von Neumann series is discussed. We present convergence guarantees for strongly-convex and smooth functions. Our analysis is much simpler in contrast to a similar…