相关论文: On the Duality Gap Convergence of ADMM Methods
Adaptive gradient methods, which adopt historical gradient information to automatically adjust the learning rate, despite the nice property of fast convergence, have been observed to generalize worse than stochastic gradient descent (SGD)…
This technical note is an ancillary material for our research paper (Itoh and Parente, 2019). We discuss an alternating direction method of multipliers with generalized augmented terms (ADMM-GAT) and introduce a generalized residual balance…
Alternating Direction Method of Multipliers (ADMM) is a widely used tool for machine learning in distributed settings, where a machine learning model is trained over distributed data sources through an interactive process of local…
We propose a new relative-error inexact version of the alternating direction method of multipliers (ADMM) for convex optimization. We prove the asymptotic convergence of our main algorithm as well as pointwise and ergodic…
Diffusion Language Models (DLMs) promise parallel generation and bidirectional context, yet they underperform autoregressive (AR) models in both likelihood modeling and generated text quality. We identify that this performance gap arises…
We discuss an online decentralized decision making problem where the agents are coupled with affine inequality constraints. Alternating Direction Method of Multipliers (ADMM) is used as the computation engine and we discuss the convergence…
The multi-block ADMM has received much attention from optimization researchers due to its excellent scalability. In this paper, the multi-block ADMM is applied to solve two large-scale problems related to isotonic regression. Numerical…
In this paper, we study the local convergence of the standard ADMM scheme for a class of nonconvex composite problems arising from modern imaging and machine learning models. This problem is constrained by a closed convex set, while its…
This paper considers the problem of distributed model fitting using the alternating directions method of multipliers (ADMM). ADMM splits the learning problem into several smaller subproblems, usually by partitioning the data samples. The…
A lift-and-permute scheme of alternating direction method of multipliers (ADMM) is proposed for linearly constrained convex programming. It contains not only the newly developed balanced augmented Lagrangian method and its dual-primal…
Adaptive gradient-descent optimizers are the standard choice for training neural network models. Despite their faster convergence than gradient-descent and remarkable performance in practice, the adaptive optimizers are not as well…
Non-convex constrained optimizations are ubiquitous in robotic applications such as multi-agent navigation, UAV trajectory optimization, and soft robot simulation. For this problem class, conventional optimizers suffer from small step sizes…
Agentic artificial intelligence (AI) -- multi-agent systems that combine large language models with external tools and autonomous planning -- are rapidly transitioning from research laboratories into high-stakes domains. Our earlier "Basic"…
This paper introduces a parallel and distributed extension to the alternating direction method of multipliers (ADMM) for solving convex problem: minimize $\sum_{i=1}^N f_i(x_i)$ subject to $\sum_{i=1}^N A_i x_i=c, x_i\in \mathcal{X}_i$. The…
Primal-Dual Hybrid Gradient (PDHG) and Alternating Direction Method of Multipliers (ADMM) are two widely-used first-order optimization methods. They reduce a difficult problem to simple subproblems, so they are easy to implement and have…
We present a unified viewpoint of proximal point method (PPM), primal-dual hybrid gradient (PDHG) and alternating direction method of multipliers (ADMM) for solving convex-concave primal-dual problems. This viewpoint shows the equivalence…
Stochastic gradient methods (SGMs) are the predominant approaches to train deep learning models. The adaptive versions (e.g., Adam and AMSGrad) have been extensively used in practice, partly because they achieve faster convergence than the…
The nonconvex and nonsmooth finite-sum optimization problem with linear constraint has attracted much attention in the fields of artificial intelligence, computer, and mathematics, due to its wide applications in machine learning and the…
In this article we develop a duality principle suitable for a large class of problems in optimization. The main result is obtained through basic tools of convex analysis and duality theory. We establish a correct relation between the…
Distributed radio interferometric calibration based on consensus optimization has been shown to improve the estimation of systematic errors in radio astronomical observations. The intrinsic continuity of systematic errors across frequency…