Related papers: Finite First Hitting Time versus Stochastic Conver…
Supported by the recent contributions in multiple branches, the first-order splitting algorithms became central for structured nonsmooth optimization. In the large-scale or noisy contexts, when only stochastic information on the smooth part…
The generalized smooth condition, $(L_{0},L_{1})$-smoothness, has triggered people's interest since it is more realistic in many optimization problems shown by both empirical and theoretical evidence. Two recent works established the…
In this work, we consider convex optimization problems with smooth objective function and nonsmooth functional constraints. We propose a new stochastic gradient algorithm, called Stochastic Halfspace Approximation Method (SHAM), to solve…
This paper proposes a novel global optimization algorithm, Particle Filter-Based Optimization (PFO), designed for a class of stochastic optimization problems in which the objective function lacks an analytical form and is subject to noisy…
Particle swarm optimization is a popular method for solving difficult optimization problems. There have been attempts to formulate the method in formal probabilistic or stochastic terms (e.g. bare bones particle swarm) with the aim to…
Solving complex real problems often demands advanced algorithms, and then continuous improvements in the internal operations of a search technique are needed. Hybrid algorithms, parallel techniques, theoretical advances, and much more are…
In this paper, we study a class of stochastic and finite-sum convex optimization problems with deterministic constraints. Existing methods typically aim to find an $\epsilon$-$expectedly\ feasible\ stochastic\ optimal$ solution, in which…
In this paper, we present a unified algorithm for stochastic optimization that makes use of a "momentum" term; in other words, the stochastic gradient depends not only on the current true gradient of the objective function, but also on the…
The paper is devoted to upper bounds on the expected first hitting times of the sets of local or global optima for non-elitist genetic algorithms with very high selection pressure. The results of this paper extend the range of situations…
The first-passage time (FPT), i.e., the moment when a stochastic process reaches a given threshold value for the first time, is a fundamental mathematical concept with immediate applications. In particular, it quantifies the statistics of…
Metaheuristic algorithms are becoming an important part of modern optimization. A wide range of metaheuristic algorithms have emerged over the last two decades, and many metaheuristics such as particle swarm optimization are becoming…
The typical particle filtering approximation error is exponentially dependent on the dimension of the model. Therefore, to control this error, an enormous number of particles are required, which means a heavy computational burden that is…
Natural systems often exhibit chaotic behavior in their space-time evolution. Systems transiting between chaos and order manifest a potential to compute, as shown with cellular automata and artificial neural networks. We demonstrate that…
The total complexity (measured as the total number of gradient computations) of a stochastic first-order optimization algorithm that finds a first-order stationary point of a finite-sum smooth nonconvex objective function $F(w)=\frac{1}{n}…
Motivated by collaborative reinforcement learning (RL) and optimization with time-correlated data, we study a generic federated stochastic approximation problem involving $M$ agents, where each agent is characterized by an agent-specific…
In noisy environments such as the cell, many processes involve target sites that are often hidden or inactive, and thus not always available for reaction with diffusing entities. To understand reaction kinetics in these situations, we study…
The growing prevalence of nonsmooth optimization problems in machine learning has spurred significant interest in generalized smoothness assumptions. Among these, the (L0, L1)-smoothness assumption has emerged as one of the most prominent.…
Local search algorithms applied to optimization problems often suffer from getting trapped in a local optimum. The common solution for this deficiency is to restart the algorithm when no progress is observed. Alternatively, one can start…
The time instant -- the first-passage time (FPT) -- when a diffusive particle (e.g., a ligand such as oxygen or a signalling protein) for the first time reaches an immobile target located on the surface of a bounded three-dimensional domain…
When designing optimal controllers for any system, it is often the case that the true state of the system is unknown to the controller, for example due to noisy measurements or partially observable states. Incomplete state information must…