相关论文: Effective Blind Source Separation Based on the Ada…

Optimization is essential in deep learning. The foundational method upon which most optimizers are built is momentum-based stochastic gradient descent. However, it suffers from two key drawbacks. First, it has noisy and varying gradients,…

This paper proposes a conjugate-gradient-based Adam algorithm blending Adam with nonlinear conjugate gradient methods and shows its convergence analysis. Numerical experiments on text classification and image classification show that the…

In this paper a novel distributed algorithm for blind macro calibration in sensor networks based on output synchronization is proposed. The algorithm is formulated as a set of gradient-type recursions for estimating parameters of sensor…

Extraction of latent sources of complex stimuli is critical for making sense of the world. While the brain solves this blind source separation (BSS) problem continuously, its algorithms remain unknown. Previous work on…

In this paper, we address a blind source separation (BSS) problem and propose a new extended framework of independent positive semidefinite tensor analysis (IPSDTA). IPSDTA is a state-of-the-art BSS method that enables us to take…

We consider distributed optimization problems in which a group of agents are to collaboratively seek the global optimum through peer-to-peer communication networks. The problem arises in various application areas, such as resource…

In a recent paper [4], Duarte and Jutten investigated the Blind Source Separation (BSS) problem, for the nonlinear mixing model that they introduced in that paper. They proposed to solve this problem by using information-theoretic tools,…

In this paper, a low complexity time domain semi-blind algorithm is proposed to estimate and track the time varying MIMO OFDM channels. First, the proposed least mean squares (LMS) based algorithm is developed for the training mode and then…

The success of deep learning can be attributed to various factors such as increase in computational power, large datasets, deep convolutional neural networks, optimizers etc. Particularly, the choice of optimizer affects the generalization,…

Adam is the important optimization algorithm to guarantee efficiency and accuracy for training many important tasks such as BERT and ImageNet. However, Adam is generally not compatible with information (gradient) compression technology.…

This work presents blind joint interference suppression and power allocation algorithms for DS-CDMA networks with multiple relays and decode and forward protocols. A scheme for joint allocation of power levels across the relays subject to…

We address a nonstationary blind source separation (BSS) problem. The model includes both nonstationary sources and mixing. Therefore, we introduce an algorithm for joint BSS and estimation of stationarity-breaking deformations and spectra.…

Source separation problems are ubiquitous in the physical sciences; any situation where signals are superimposed calls for source separation to estimate the original signals. In this tutorial I will discuss the Bayesian approach to the…

We observe that the traditional use of DP with the Adam optimizer introduces a bias in the second moment estimation, due to the addition of independent noise in the gradient computation. This bias leads to a different scaling for low…

We introduce MADGRAD, a novel optimization method in the family of AdaGrad adaptive gradient methods. MADGRAD shows excellent performance on deep learning optimization problems from multiple fields, including classification and…

This paper proposes a novel memetic algorithm (MA) for the blind equalization of digital multiuser channels with Direct-Sequence / Code-Division Multiple-Access (DS/CDMA) sharing scheme. Equalization involves two different tasks, the…

Despite the popularity of the Adam optimizer in practice, most theoretical analyses study Stochastic Gradient Descent (SGD) as a proxy for Adam, and little is known about how the solutions found by Adam differ. In this paper, we show that…

We propose a new algorithm for blind source separation (BSS) using independent vector analysis (IVA). This is an improvement over the popular auxiliary function based IVA (AuxIVA) with iterative projection (IP) or iterative source steering…

In this paper, we provide a rigorous proof of convergence of the Adaptive Moment Estimate (Adam) algorithm for a wide class of optimization objectives. Despite the popularity and efficiency of the Adam algorithm in training deep neural…

Modern large-scale finite-sum optimization relies on two key aspects: distribution and stochastic updates. For smooth and strongly convex problems, existing decentralized algorithms are slower than modern accelerated variance-reduced…