Related papers: Adaptive Momentum via Minimal Dual Function for Ac…
We develop an adaptive-metric framework for norm-minimization-based outer approximation algorithms in bounded convex vector optimization. The key idea is to let the scalarization metric vary across iterations while measuring approximation…
In this paper, we propose a federated algorithm for solving large linear systems that is inspired by the classic randomized Kaczmarz algorithm. We provide convergence guarantees of the proposed method, and as a corollary of our analysis, we…
Randomized Kaczmarz methods form a family of linear system solvers which converge by repeatedly projecting their iterates onto randomly sampled equations. While effective in some contexts, such as highly over-determined least squares,…
Accurate estimation of the dynamic states of a synchronous machine (e.g., rotor s angle and speed) is essential in monitoring and controlling transient stability of a power system. It is well known that the covariance matrixes of process…
Through theoretical and experimental validation, unlike all existing adaptive methods like Adam which penalize frequently-changing parameters and are only applicable to sparse gradients, we propose the simplest SGD enhanced method,…
It is well-known that the high computational complexity and the insufficient samples in large-scale array signal processing restrict the real-world applications of the conventional full-dimensional adaptive beamforming (sample matrix…
For solving pseudo-convex global optimization problems, we present a novel fully adaptive steepest descent method (or ASDM) without any hard-to-estimate parameters. For the step-size regulation in an $\varepsilon$-normalized direction, we…
The Kaczmarz method is an efficient iterative algorithm for large-scale linear systems. However, its linear convergence rate suffers from ill-conditioned problems and is highly sensitive to the smallest nonzero singular value. In this work,…
In this paper, we propose, analyze and demonstrate a dynamic momentum method to accelerate power and inverse power iterations with minimal computational overhead. The method can be applied to real diagonalizable matrices, is provably…
Recovery of a sparse signal from a nonlinear system arises in many practical applications including compressive sensing, image reconstruction and machine learning. In this paper, a fast block nonlinear Bregman-Kaczmarz method with averaging…
The scope of this study is to develop a novel sensorless adaptive vibration suppression controller for two-mass systems with joint estimation of states and controller parameters. Unlike existing solutions, we simultaneously: (i) propose an…
We present a randomized iterative algorithm that exponentially converges in expectation to the minimum Euclidean norm least squares solution of a given linear system of equations. The expected number of arithmetic operations required to…
The Kaczmarz method (KZ) and its variants, which are types of stochastic gradient descent (SGD) methods, have been extensively studied due to their simplicity and efficiency in solving linear equation systems. The iterative thresholding…
In this paper, for solving nonlinear systems we propose two pseudoinverse-free greedy block methods with momentum by combining the residual-based weighted nonlinear Kaczmarz and heavy ball methods. Without the full column rank assumptions…
The Randomized Kaczmarz Algorithm is a randomized method which aims at solving a consistent system of over determined linear equations. This note discusses how to find an optimized randomization scheme for this algorithm, which is related…
Broadband wireless channels usually have the sparse nature. Based on the assumption of Gaussian noise model, adaptive filtering algorithms for reconstruction sparse channels were proposed to take advantage of channel sparsity. However,…
In this paper, we devise a sparse array design algorithm for adaptive beamforming. Our strategy is based on finding a sparse beamformer weight to maximize the output signal-to-interference-plus-noise ratio (SINR). The proposed method…
The vast majority of modern deep learning models are trained with momentum-based first-order optimizers. The momentum term governs the optimizer's memory by determining how much each past gradient contributes to the current convergence…
The generalized Gearhart-Koshy acceleration is a recent exact affine search technique designed for the method of cyclic projections onto hyperplanes, i.e., the Kaczmarz method. However, its convergence properties, particularly the linear…
Bayesian curve fitting plays an important role in inverse problems, and is often addressed using the Reversible Jump Markov Chain Monte Carlo (RJMCMC) algorithm. However, this algorithm can be computationally inefficient without…