Related papers: Is perturbation an effective restart strategy?
We introduce the profligacy of a search process as a competition between its expected cost and the probability of finding the target. The arbiter of the competition is a parameter $\lambda$ that represents how much a searcher invests into…
Despite the tremendous success of deep neural networks in various learning problems, it has been observed that adding an intentionally designed adversarial perturbation to inputs of these architectures leads to erroneous classification with…
We develop new perturbation techniques for conducting convergence analysis of various first-order algorithms for a class of nonsmooth optimization problems. We consider the iteration scheme of an algorithm to construct a perturbed…
We introduce a new class of first passage time optimization driven by threshold resetting, inspired by many natural processes where crossing a critical limit triggers failure, degradation or transition. In here, search agents are…
We propose the use of controlled perturbations to address the challenging question of optimal active-set prediction for interior point methods. Namely, in the context of linear programming, we consider perturbing the inequality…
We consider global optimization problems, where the feasible region $\X$ is a compact subset of $\mathbb{R}^d$ with $d \geq 10$. For these problems, we demonstrate the following. First: the actual convergence of global random search…
Restarting a stochastic search process can accelerate its completion by providing an opportunity to take a more favorable path with each reset. This strategy, known as stochastic resetting, is well studied in random processes. Here, we…
Path-following algorithms are frequently used in composite optimization problems where a series of subproblems, with varying regularization hyperparameters, are solved sequentially. By reusing the previous solutions as initialization,…
We revisit the Reinforce policy gradient algorithm from the literature. Note that this algorithm typically works with cost returns obtained over random length episodes obtained from either termination upon reaching a goal state (as with…
A plethora of attack methods have been proposed to generate adversarial examples, among which the iterative methods have been demonstrated the ability to find a strong attack. However, the computation of an adversarial perturbation for a…
Randomized search algorithms for hard combinatorial problems exhibit a large variability of performances. We study the different types of rare events which occur in such out-of-equilibrium stochastic processes and we show how they cooperate…
While neural architecture search methods have been successful in previous years and led to new state-of-the-art performance on various problems, they have also been criticized for being unstable, being highly sensitive with respect to their…
Recurrent neural network (RNN)'s architecture is a key factor influencing its performance. We propose algorithms to optimize hidden sizes under running time constraint. We convert the discrete optimization into a subset selection problem.…
Recently noticed ability of restart to reduce the expected completion time of first-passage processes allows appealing opportunities for performance improvement in a variety of settings. However, complex stochastic processes often exhibit…
We recently presented a methodology for quantitatively reducing the risk and cost of executing electronic transactions in a bursty network environment such as the Internet. In the language of portfolio theory, time to complete a transaction…
Imbalanced problems can arise in different real-world situations, and to address this, certain strategies in the form of resampling or balancing algorithms are proposed. This issue has largely been studied in the context of classification,…
We find an advantage of recombination for a category of complex fitness landscapes. Recent studies of empirical fitness landscapes reveal complex gene interactions and multiple peaks, and recombination can be a powerful mechanism for…
In this paper we study time-dependent scheduling problems where activities consume a resource with limited availability. Activity durations as well as resource consumptions are assumed to be time-dependent and the resource can be…
The mean completion time of a stochastic process may be rendered finite and minimised by a judiciously chosen restart protocol, which may either be stochastic or deterministic. Here we study analytically an arbitrary stochastic search…
In this paper we demonstrate a simple heuristic adaptive restart technique that can dramatically improve the convergence rate of accelerated gradient schemes. The analysis of the technique relies on the observation that these schemes…