Related papers: A Momentum Two-gradient Direction Algorithm with V…
Multiple-stage adaptive architectures are conceived to face with the problem of target detection buried in noise, clutter, and intentional interference. First, a scenario where the radar system is under the electronic attack of noise-like…
Gradient-free prompt optimization methods have made significant strides in enhancing the performance of closed-source Large Language Models (LLMs) across a wide range of tasks. However, existing approaches make light of the importance of…
Variational algorithms are a promising paradigm for utilizing near-term quantum devices for modeling electronic states of molecular systems. However, previous bounds on the measurement time required have suggested that the application of…
This paper proposes an original solution to input saturation and dead zone of fractional order system. To overcome these nonsmooth nonlinearities, the control input is decomposed into two independent parts by introducing an intermediate…
The training of deep neural networks is inherently a nonconvex optimization problem, yet standard approaches such as stochastic gradient descent (SGD) require simultaneous updates to all parameters, often leading to unstable convergence and…
Using double-smoothing technique and stochastic mirror descent with inexact oracle we built an optimal algorithm (up to a multiplicative factor) for two-points gradient-free non-smooth stochastic convex programming. We investigate how much…
Two major momentum-based techniques that have achieved tremendous success in optimization are Polyak's heavy ball method and Nesterov's accelerated gradient. A crucial step in all momentum-based methods is the choice of the momentum…
The close relationship between the feedforward ANC system and the stereo acoustic echo cancellation system is revealed in this paper. Accordingly, the convergence behavior of the ANC system can be analyzed by investigating the joint…
Navigating toward a known target in a noisy environment is a fundamental problem shared across biological, physical, and engineered systems. Although optimal strategies are often framed in terms of continuous, fine-grained feedback, we show…
Active noise control (ANC) has become popular for reducing noise and thus enhancing user comfort in headphones. While feedback control offers an effective way to implement ANC, it is restricted by uncertainty of the controlled system that…
Recently, Stochastic Gradient Descent (SGD) and its variants have become the dominant methods in the large-scale optimization of machine learning (ML) problems. A variety of strategies have been proposed for tuning the step sizes, ranging…
Normalization techniques are a boon for modern deep learning. They let weights converge more quickly with often better generalization performances. It has been argued that the normalization-induced scale invariance among the weights…
Quantum computing, which has the power to accelerate many computing applications, is currently a technology under development. As a result, the existing noisy intermediate-scale quantum (NISQ) computers suffer from different hardware noise…
The Adaptive Momentum Estimation (Adam) algorithm is highly effective in training various deep learning tasks. Despite this, there's limited theoretical understanding for Adam, especially when focusing on its vanilla form in non-convex…
We analyze the convergence properties of a modified barrier method for solving bound-constrained optimization problems where evaluations of the objective function and its derivatives are affected by bounded and non-diminishing noise. The…
We study how to safely control nonlinear control-affine systems that are corrupted with bounded non-stochastic noise, i.e., noise that is unknown a priori and that is not necessarily governed by a stochastic model. We focus on safety…
Leveraging machine learning methods to solve constraint satisfaction problems has shown promising, but they are mostly limited to a static situation where the problem description is completely known and fixed from the beginning. In this…
This paper presents a novel accelerated distributed algorithm for unconstrained consensus optimization over static undirected networks. The proposed algorithm combines the benefits of acceleration from momentum, the robustness of the…
In this paper, we establish the almost sure convergence of two-timescale stochastic gradient descent algorithms in continuous time under general noise and stability conditions, extending well known results in discrete time. We analyse…
Recently, video recognition is emerging with the help of multi-modal learning, which focuses on integrating distinct modalities to improve the performance or robustness of the model. Although various multi-modal learning methods have been…