Related papers: Convergence Analysis for Restarted Anderson Mixing…
Recent progress has been made with Adaptive Multiple Importance Sampling (AMIS) methods that show improvement in effective sample size. However, consistency for the AMIS estimator has only been established in very restricted cases.…
A lift-and-permute scheme of alternating direction method of multipliers (ADMM) is proposed for linearly constrained convex programming. It contains not only the newly developed balanced augmented Lagrangian method and its dual-primal…
Thanks to its versatility, its simplicity, and its fast convergence, ADMM is among the most widely used approaches for solving a convex problem in distributed form. However, making it running efficiently is an art that requires a fine…
Mixed linear regression involves the recovery of two (or more) unknown vectors from unlabeled linear measurements; that is, where each sample comes from exactly one of the vectors, but we do not know which one. It is a classic problem, and…
The performance of many hard combinatorial problem solvers depends strongly on their parameter settings, and since manual parameter tuning is both tedious and suboptimal the AI community has recently developed several algorithm…
Adaptive Moment Estimation (ADAM) is a very popular training algorithm for deep neural networks and belongs to the family of adaptive gradient descent optimizers. However to the best of the authors knowledge no complete convergence analysis…
In this overview article we will consider the deliberate restarting of algorithms, a meta technique, in order to improve the algorithm's performance, e.g., convergence rates or approximation guarantees. One of the major advantages is that…
This paper proposes an accelerated method for approximately solving partially observable Markov decision process (POMDP) problems offline. Our method carefully combines two existing tools: Anderson acceleration (AA) and the fast informed…
Optimization plays a key role in machine learning. Recently, stochastic second-order methods have attracted much attention due to their low computational cost in each iteration. However, these algorithms might perform poorly especially if…
In this paper, we study the robust linearization of nonlinear poromechanics of unsaturated materials. The model of interest couples the Richards equation with linear elasticity equations, employing the equivalent pore pressure. In practice…
Convergence rates are established for an inexact accelerated alternating direction method of multipliers (I-ADMM) for general separable convex optimization with a linear constraint. Both ergodic and non-ergodic iterates are analyzed.…
Purpose: To propose an alternating learning approach to learn the sampling pattern (SP) and the parameters of variational networks (VN) in accelerated parallel magnetic resonance imaging (MRI). Methods: The approach alternates between…
In this paper we develop convergence and acceleration theory for Anderson acceleration applied to Newton's method for nonlinear systems in which the Jacobian is singular at a solution. For these problems, the standard Newton algorithm…
With the rapid development of deep learning, automatic modulation recognition (AMR), as an important task in cognitive radio, has gradually transformed from traditional feature extraction and classification to automatic classification by…
This paper primarily presents numerical results for the Anderson accelerated Newton method on a set of benchmark problems. The results demonstrate superlinear convergence to solutions of both degenerate and nondegenerate problems. The…
Iteratively reweighted L1 (IRL1) algorithm is a common algorithm for solving sparse optimization problems with nonconvex and nonsmooth regularization. The development of its acceleration algorithm, often employing Nesterov acceleration, has…
This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Numerous renowned algorithms for tackling the compressed sensing problem…
Several new accelerated methods in minimax optimization and fixed-point iterations have recently been discovered, and, interestingly, they rely on a mechanism distinct from Nesterov's momentum-based acceleration. In this work, we show that…
In this paper, we introduce a new iteration method and show that this iteration method can be used to approximate fixed point of almost contraction mappings. Furthermore, we prove that the new iteration method is equivalent to both Mann…
Adam is a widely used stochastic optimization method for deep learning applications. While practitioners prefer Adam because it requires less parameter tuning, its use is problematic from a theoretical point of view since it may not…