Related papers: Optimal step-size of least mean absolute fourth al…

In this paper, we investigate the problem of stochastic multi-level compositional optimization, where the objective function is a composition of multiple smooth but possibly non-convex functions. Existing methods for solving this problem…

The training of deep neural networks is inherently a nonconvex optimization problem, yet standard approaches such as stochastic gradient descent (SGD) require simultaneous updates to all parameters, often leading to unstable convergence and…

We consider the minimization of a sum of an expectation-valued coordinate-wise $L_i$-smooth nonconvex function and a nonsmooth block-separable convex regularizer. We propose an asynchronous variance-reduced algorithm, where in each…

To fast approximate maximum likelihood estimators with massive data, this paper studies the Optimal Subsampling Method under the A-optimality Criterion (OSMAC) for generalized linear models. The consistency and asymptotic normality of the…

Symmetric Nonnegative Matrix Factorization (SNMF) models arise naturally as simple reformulations of many standard clustering algorithms including the popular spectral clustering method. Recent work has demonstrated that an elementary…

The diffusion least-mean square (dLMS) algorithms have attracted much attention owing to its robustness for distributed estimation problems. However, the performance of such filters may change when they are implemented for suppressing…

In this paper, we propose a provably correct algorithm for convolutive nonnegative matrix factorization (CNMF) under separability assumptions. CNMF is a convolutive variant of nonnegative matrix factorization (NMF), which functions as an…

Constrained adaptive filtering algorithms inculding constrained least mean square (CLMS), constrained affine projection (CAP) and constrained recursive least squares (CRLS) have been extensively studied in many applications. Most existing…

We investigate a modified split-step Fourier method (SSFM) by including low-pass filters in the linear steps. This method can simultaneously achieve a higher simulation accuracy and a slightly reduced complexity.

Tuning hyperparameters, such as the stepsize, presents a major challenge of training machine learning models. To address this challenge, numerous adaptive optimization algorithms have been developed that achieve near-optimal complexities,…

Despite the rapid development of large language models (LLMs), a fundamental challenge persists: the lack of high-quality optimization modeling datasets hampers LLMs' robust modeling of practical optimization problems from natural language…

We propose a block least mean square (LMS) algorithm to monitor the longitudinal power profile of a fiber-optic link through receiver-based digital data from a coherent detector. Compared to the benchmark least squares (LS) method, the…

In this paper, we have evaluated various methods of time-frequency-selective fading channels estimation in OFDM system and some of them improved under time varying conditions. So, these different techniques will be studied through different…

Many computer vision problems can be posed as learning a low-dimensional subspace from high dimensional data. The low rank matrix factorization (LRMF) represents a commonly utilized subspace learning strategy. Most of the current LRMF…

In orthogonal frequency division multiplexing with index modulation (OFDM-IM), the active subcarriers can convey information bits by modulated symbols as well as their indices. OFDM-IM has attracted a great deal of attention from…

Approximate dynamic programming (ADP) has proven itself in a wide range of applications spanning large-scale transportation problems, health care, revenue management, and energy systems. The design of effective ADP algorithms has many…

We propose a version of least-mean-square (LMS) algorithm for sparse system identification. Our algorithm called online linearized Bregman iteration (OLBI) is derived from minimizing the cumulative prediction error squared along with an…

Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS). The Gaussian kernel is usually the default kernel in KAF algorithms, but selecting the proper kernel size…

The purpose of this work is to introduce a new idea of how to avoid the factorization of large matrices during the solution of stiff systems of ODEs. Starting from the general form of an explicit linear multistep method we suggest to…

Partial diffusion scheme is an effective method for reducing computational load and power consumption in adaptive network implementation. The Information is exchanged among the nodes, usually over noisy links. In this paper, we consider a…