Related papers: Approximate analysis of search algorithms with "ph…

We propose a novel optimization algorithm for continuous functions using geodesics and contours under conformal mapping.The algorithm can find multiple optima by first following a geodesic curve to a local optimum then traveling to the next…

Randomized search algorithms for hard combinatorial problems exhibit a large variability of performances. We study the different types of rare events which occur in such out-of-equilibrium stochastic processes and we show how they cooperate…

We recast Grover's generalised search algorithm in a geometric language even when the states are not approximately orthogonal. We provide a possible search algorithm based on an arbitrary unitary transformation which can speed up the steps…

Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous. In engineering applications, the objective function is typically calculated with a numerically costly black-box…

In this work, we address unconstrained finite-sum optimization problems, with particular focus on instances originating in large scale deep learning scenarios. Our main interest lies in the exploration of the relationship between recent…

Variational inequalities are a formalism that includes games, minimization, saddle point, and equilibrium problems as special cases. Methods for variational inequalities are therefore universal approaches for many applied tasks, including…

Stochastic coordinate descent algorithms are efficient methods in which each iterate is obtained by fixing most coordinates at their values from the current iteration, and approximately minimizing the objective with respect to the remaining…

Despite significant advances in characterizing the highly nonconvex landscapes of constraint satisfaction problems, the good performance of certain algorithms in solving hard combinatorial optimization tasks remains poorly understood. This…

Safe learning and optimization deals with learning and optimization problems that avoid, as much as possible, the evaluation of non-safe input points, which are solutions, policies, or strategies that cause an irrecoverable loss (e.g.,…

Understanding the behavior of numerical metaheuristic optimization algorithms is critical for advancing their development and application. Traditional visualization techniques, such as convergence plots, trajectory mapping, and fitness…

In this paper a novel stochastic optimization and extremum seeking algorithm is presented, one which is based on time-delayed random perturbations and step size adaptation. For the case of a one-dimensional quadratic unconstrained…

The optimization of dynamic problems is both widespread and difficult. When conducting dynamic optimization, a balance between reinitialization and computational expense has to be found. There are multiple approaches to this. In parallel…

There is a clear need for efficient algorithms to tune hyperparameters for statistical learning schemes, since the commonly applied search methods (such as grid search with N-fold cross-validation) are inefficient and/or approximate.…

Large scale numerical experiments are commonplace today in theoretical physics. The high performance algorithms described herein are the most compact, efficient methods known for representing and analyzing systems modeled well by sets or…

Finding global optima in high-dimensional optimization problems is extremely challenging since the number of function evaluations required to sufficiently explore the search space increases exponentially with its dimensionality.…

We consider an agent seeking to obtain an item, potentially available at different locations in a physical environment. The traveling costs between locations are known in advance, but there is only probabilistic knowledge regarding the…

We develop a system-theoretic framework for the structured analysis of distributed optimization algorithms with decomposable cost functions. We model such algorithms as a network of interacting dynamical systems and derive tests for…

Optimizing paths on networks is crucial for many applications, from subway traffic to Internet communication. As global path optimization that takes account of all path-choices simultaneously is computationally hard, most existing routing…

We study optimization algorithms for the finite sum problems frequently arising in machine learning applications. First, we propose novel variants of stochastic gradient descent with a variance reduction property that enables linear…

We combine beam search with the probabilistic pruning technique of nucleus sampling to create two deterministic nucleus search algorithms for natural language generation. The first algorithm, p-exact search, locally prunes the next-token…