Related papers: Consensus-Based Optimization for Saddle Point Prob…
We analyze the Consensus-Based Optimization (CBO) algorithm with a consensus point rescaled by a small fixed parameter $\kappa \in (0,1)$. Under minimal assumptions on the objective function and the initial data, we establish its…
We propose an approach to saddle point optimization relying only on oracles that solve minimization problems approximately. We analyze its convergence property on a strongly convex--concave problem and show its linear convergence toward the…
Population-based methods are often used to solve multimodal optimization problems. By combining niching or clustering strategy, the state-of-the-art approaches generally divide the population into several subpopulations to find multiple…
We study the derivative-free global optimization algorithm Consensus-Based Optimization (CBO), establishing uniform-in-time propagation of chaos as well as an almost uniform-in-time stability result for the microscopic particle system.…
We propose an algorithm to approximate solutions of global optimization problems in Sobolev spaces that follows the spirit of Consensus-based algorithms in finite dimensions. The main ingredient are Gaussian processes. In fact, we exploit…
Neural combinatorial optimization (NCO) has gained significant attention due to the potential of deep learning to efficiently solve combinatorial optimization problems. NCO has been widely applied to job shop scheduling problems (JSPs) with…
This study proposes a method for estimating the mechanical parameters of vehicles and bridges and the road unevenness, using only vehicle vibration and position data. In the proposed method, vehicle input and bridge vibration are estimated…
In this paper, we study the conditional stochastic optimization (CSO) problem which covers a variety of applications including portfolio selection, reinforcement learning, robust learning, causal inference, etc. The sample-averaged gradient…
Convolved Gaussian Process (CGP) is able to capture the correlations not only between inputs and outputs but also among the outputs. This allows a superior performance of using CGP than standard Gaussian Process (GP) in the modelling of…
Particle swarm optimization (PSO) is a widely used nature-inspired meta-heuristic for solving continuous optimization problems. However, when running the PSO algorithm, one encounters the phenomenon of so-called stagnation, that means in…
In this paper, we investigate a class of constrained saddle point (SP) problems where the objective function is nonconvex-concave and smooth. This class of problems has wide applicability in machine learning, including robust multi-class…
We extend the Frank-Wolfe (FW) optimization algorithm to solve constrained smooth convex-concave saddle point (SP) problems. Remarkably, the method only requires access to linear minimization oracles. Leveraging recent advances in FW…
Contextual Stochastic Bilevel Optimization (CSBO) extends standard stochastic bilevel optimization (SBO) by incorporating context-dependent lower-level problems. CSBO problems are generally intractable since existing methods require solving…
Bayesian optimization is a class of data efficient model based algorithms typically focused on global optimization. We consider the more general case where a user is faced with multiple problems that each need to be optimized conditional on…
In this paper, we focus on solving a class of constrained non-convex non-concave saddle point problems in a decentralized manner by a group of nodes in a network. Specifically, we assume that each node has access to a summand of a global…
In this paper, we study the problem of escaping from saddle points in smooth nonconvex optimization problems subject to a convex set $\mathcal{C}$. We propose a generic framework that yields convergence to a second-order stationary point of…
Black-box optimization algorithms have been widely used in various machine learning problems, including reinforcement learning and prompt fine-tuning. However, directly optimizing the training loss value, as commonly done in existing…
We propose a novel method for sampling and optimization tasks based on a stochastic interacting particle system. We explain how this method can be used for the following two goals: (i) generating approximate samples from a given target…
The global optimization of a high-dimensional black-box function under black-box constraints is a pervasive task in machine learning, control, and engineering. These problems are challenging since the feasible set is typically non-convex…
We consider convex-concave saddle point problems with a separable structure and non-strongly convex functions. We propose an efficient stochastic block coordinate descent method using adaptive primal-dual updates, which enables flexible…