Related papers: Mean-performance of sharp restart I: Statistical r…

We consider a combined restarting and adaptive backtracking strategy for the popular Fast Iterative Shrinking-Thresholding Algorithm frequently employed for accelerating the convergence speed of large-scale structured convex optimization…

Classical first passage under resetting is a paradigm in the search process. Despite its multitude of applications across interdisciplinary sciences, experimental realizations of such resetting processes posit practical challenges in…

We observe a trend regarding restart strategies used in SAT solvers. A few years ago, most state-of-the-art solvers restarted on average after a few thousands of backtracks. Currently, restarting after a dozen backtracks results in much…

Stochastic restoration algorithms allow to explore the space of solutions that correspond to the degraded input. In this paper we reveal additional fundamental advantages of stochastic methods over deterministic ones, which further motivate…

For obtaining optimal first-order convergence guarantee for stochastic optimization, it is necessary to use a recurrent data sampling algorithm that samples every data point with sufficient frequency. Most commonly used data sampling…

We introduce a new restarting scheme for a continuous inertial dynamics with Hessian driven-damping, and establish a linear convergence rate for the function values along the restarted trajectories. The proposed routine is implemented…

A new model of search based on stochastic resetting is introduced, wherein rate of resets depends explicitly on time elapsed since the beginning of the process. It is shown that rate inversely proportional to time leads to paradoxical…

The prioritization of restoration actions after large power system outages plays a key role in how quickly power can be restored. It has been shown that fast and intuitive heuristics for restoration prioritization most often result in…

A class of restarted randomized surrounding methods are presented to accelerate the surrounding algorithms by restarted techniques for solving the linear equations. Theoretical analysis prove that the proposed method converges under the…

A general rate estimation method is proposed that is based on studying the in-sample evolution of appropriately chosen diverging/converging statistics. The proposed rate estimators are based on simple least squares arguments, and are shown…

First-passage processes can be divided in two classes: those that are accelerated by the introduction of restart and those that display an opposite response. In physical systems, a transition between the two classes may occur as governing…

We propose a new iterative procedure to optimize the restart for meta-heuristic algorithms to solve combinatorial optimization, which uses independent algorithm executions. The new procedure consists of either adding new executions or…

We present a simple scheme for restarting first-order methods for convex optimization problems. Restarts are made based only on achieving specified decreases in objective values, the specified amounts being the same for all optimization…

``When in a difficult situation, it is sometimes better to give up and start all over again''. While this empirical truth has been regularly observed in a wide range of circumstances, quantifying the effectiveness of such a heuristic…

Hyperparameter optimization is crucial for obtaining peak performance of machine learning models. The standard protocol evaluates various hyperparameter configurations using a resampling estimate of the generalization error to guide…

Gradient restarting has been shown to improve the numerical performance of accelerated gradient methods. This paper provides a mathematical analysis to understand these advantages. First, we establish global linear convergence guarantees…

Performance modeling is a key issue in queuing theory and operation research. It is well-known that the length of a queue that awaits service or the time spent by a job in a queue depends not only on the service rate, but also crucially on…

A key trait of stochastic optimizers is that multiple runs of the same optimizer in attempting to solve the same problem can produce different results. As a result, their performance is evaluated over several repeats, or runs, on the…

First passage under restart has recently emerged as a conceptual framework suitable for the description of a wide range of phenomena, but the endless variety of ways in which restart mechanisms and first passage processes mix and match…

We introduce a new class of first passage time optimization driven by threshold resetting, inspired by many natural processes where crossing a critical limit triggers failure, degradation or transition. In here, search agents are…