Related papers: Norm-based convergence bounds for nonsymmetric alg…
We provide a new proof of the linear convergence of the alternating direction method of multipliers (ADMM) when one of the objective terms is strongly convex. Our proof is based on a framework for analyzing optimization algorithms…
Regressions created from experimental or simulated data enable the construction of metamodels, widely used in a variety of engineering applications. Many engineering problems involve multi-parametric physics whose corresponding…
We investigate the effect of non-symmetric relatively bounded perturbations on the spectrum of self-adjoint operators. In particular, we establish stability theorems for one or infinitely many spectral gaps along with corresponding…
Metric data plays an important role in various settings such as metric-based indexing, clustering, classification, and approximation algorithms in general. Due to measurement error, noise, or an inability to completely gather all the data,…
Consider the $n!$ different unitary matrices that permute $n$ $d$-dimensional quantum systems. If $d\geq n$ then they are linearly independent. This paper discusses a sense in which they are approximately orthogonal (with respect to the…
The paper introduces unbounded antilinear operators on Hilbert spaces and develops their fundamental theory. In particular, we establish a closed range theorem, a polar decomposition theorem, and the convexity of the numerical range for…
Consider a complex Hilbert space $\left(\mathcal{H}, \langle \cdot, \cdot \rangle\right)$ equipped with a positive bounded linear operator $A$ on $\mathcal{H}$. This induces a semi-norm $\|\cdot\|_A$ through the semi-inner product $\langle…
The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uniformly bounded. While this might hold for some loss functions, it is violated for cases where the objective…
Some new characterizations of nonnegative Hamiltonian operator matrices are given. Several necessary and sufficient conditions for an unbounded nonnegative Hamiltonian operators to be invertible are obtained; so that the main results in the…
Ample empirical evidence in deep neural network training suggests that a variety of optimizers tend to find nearly global optima. In this article, we adopt the reversed perspective that convergence to an arbitrary point is assumed rather…
Despite their overwhelming capacity to overfit, deep neural networks trained by specific optimization algorithms tend to generalize well to unseen data. Recently, researchers explained it by investigating the implicit regularization effect…
We develop and analyze a broad family of stochastic/randomized algorithms for inverting a matrix. We also develop specialized variants maintaining symmetry or positive definiteness of the iterates. All methods in the family converge…
We propose in this paper a unifying scheme for several algorithms from the literature dedicated to the solving of monotone inclusion problems involving compositions with linear continuous operators in infinite dimensional Hilbert spaces. We…
Two-grid theory plays a fundamental role in the design and analysis of multigrid methods. This paper is devoted to a new convergence analysis of two-grid methods for singular and symmetric positive semidefinite systems. Specifically, we…
Linearized alternating direction method of multipliers (ADMM) as an extension of ADMM has been widely used to solve linearly constrained problems in signal processing, machine leaning, communications, and many other fields. Despite its…
This paper proves joint convergence of the approximation error for several stochastic integrals with respect to local Brownian semimartingales, for nonequidistant and random grids. The conditions needed for convergence are that the Lebesgue…
An oriented hypergraph is an oriented incidence structure that extends the concepts of signed graphs, balanced hypergraphs, and balanced matrices. We introduce hypergraphic structures and techniques that generalize the circuit…
We consider linear inverse problems in a nonparametric statistical framework. Both the signal and the operator are unknown and subject to error measurements. We establish minimax rates of convergence under squared error loss when the…
Let $ \mathbb{B}(\mathscr{H})$ represent the $C^*$-algebra, which consists of all bounded linear operators on $\mathscr{H},$ and let $N ( .) $ be a norm on $ \mathbb{B}(\mathscr{H})$. We define a norm $w_{(N,e)} (. , . )$ on $…
Consider $n$ agents connected over a network collaborating to minimize the average of their local cost functions combined with a common nonsmooth function. This paper introduces a unified algorithmic framework for solving such a problem…