Related papers: Revisiting a Fast Newton Solver for a 2-D Spectral…
We apply the approach developed in our previous papers to obtain examples of solutions to the inverse spectral problem (ISP) for the canonical Hamiltonian system. One of our goals is to illustrate connections of ISP with classical tools of…
The training of two-layer neural networks with nonlinear activation functions is an important non-convex optimization problem with numerous applications and promising performance in layerwise deep learning. In this paper, we develop exact…
This paper studies second-order methods for convex-concave minimax optimization. Monteiro and Svaiter (2012) proposed a method to solve the problem with an optimal iteration complexity of $\mathcal{O}(\epsilon^{-3/2})$ to find an…
Large language models (LLMs), such as ChatGPT and GPT4, have shown outstanding performance in many human life task. Attention computation plays an important role in training LLMs. Softmax unit and ReLU unit are the key structure in…
Since the number of incident energies is limited, it is difficult to directly acquire hyperspectral images (HSI) with high spatial resolution. Considering the high dimensionality and correlation of HSI, super-resolution (SR) of HSI remains…
We propose a communication efficient quasi-Newton method for large-scale multi-agent convex composite optimization. We assume the setting of a network of agents that cooperatively solve a global minimization problem with strongly convex…
Thompson's partition of a cyclic subnormal operator into normal and completely non-normal components is combined with a non-commutative calculus for hyponormal operators for separating outliers from the cloud, in rather general point…
This paper introduces an efficient algorithm for computing the best approximation of a given matrix onto the intersection of linear equalities, inequalities and the doubly nonnegative cone (the cone of all positive semidefinite matrices…
We consider the non-parametric maximum likelihood estimation in the class of Polya frequency functions of order two, viz. the densities with a concave logarithm. This is a subclass of unimodal densities and fairly rich in general. The NPMLE…
In this paper, we establish an a priori second-order estimate for admissible solutions satisfying a dynamic plurisubharmonic condition to equations involving sums of Hessian operators on compact Hermitian manifolds. The estimate is derived…
A class of second-order algorithms is proposed for minimizing smooth nonconvex functions that alternates between regularized Newton and negative curvature steps in an iteration-dependent subspace. In most cases, the Hessian matrix is…
In \cite{wang2023towards}, a dual-consistent dual-weighted residual-based $h$-adaptive method has been proposed based on a Newton-GMG framework, towards the accurate calculation of a given quantity of interest from Euler equations. The…
Second order information is useful in many ways in smooth optimization problems, including for the design of step size rules and descent directions, or the analysis of the local properties of the objective functional. However, the…
We present a refinement of the Spectral Method by incorporating an optimization method into it and generalize it to two space dimensions. We then apply this Refined Spectral Method as an extremely accurate technique for finding the bound…
Toeplitz Neural Networks (TNNs) have exhibited outstanding performance in various sequence modeling tasks. They outperform commonly used Transformer-based models while benefiting from log-linear space-time complexities. On the other hand,…
We study the optimization problem associated with fitting two-layer ReLU neural networks with respect to the squared loss, where labels are generated by a target network. We make use of the rich symmetry structure to develop a novel set of…
This paper studies convex quadratic minimization problems in which each continuous variable is coupled with a binary indicator variable. We focus on the structured setting where the Hessian matrix of the quadratic term is positive definite…
We analyze the inverse problem to reconstruct the shape of a three dimensional homogeneous dielectric obstacle from the knowledge of noisy far field data. The forward problem is solved by a system of second kind boundary integral equations.…
High-dimensional linear regression under heavy-tailed noise or outlier corruption is challenging, both computationally and statistically. Convex approaches have been proven statistically optimal but suffer from high computational costs,…
We propose a new randomized optimization method for high-dimensional problems which can be seen as a generalization of coordinate descent to random subspaces. We show that an adaptive sampling strategy for the random subspace significantly…