Related papers: On a numerical construction of doubly stochastic m…
The essential difficulty of gradient-based bilevel optimization using implicit differentiation is to estimate the inverse Hessian vector product with respect to neural network parameters. This paper proposes to tackle this problem by the…
Stochastic alternating algorithms for bi-objective optimization are considered when optimizing two conflicting functions for which optimization steps have to be applied separately for each function. Such algorithms consist of applying a…
This work aims at estimating inverse autocovariance matrices of long memory processes admitting a linear representation. A modified Cholesky decomposition is used in conjunction with an increasing order autoregressive model to achieve this…
Weight-balanced and doubly stochastic digraphs are two classes of digraphs that play an essential role in a variety of cooperative control problems, including formation control, distributed averaging, and optimization. We refer to a digraph…
We address distributed learning problems, both nonconvex and convex, over undirected networks. In particular, we design a novel algorithm based on the distributed Alternating Direction Method of Multipliers (ADMM) to address the challenges…
The inverse eigenvalue problem for real symmetric matrices of the form 0 0 0 . 0 0 * 0 0 0 . 0 * * 0 0 0 . * * 0 . . . . . . . 0 0 * . 0 0 0 0 * * . 0 0 0 * * 0 . 0 0 0 is solved. The solution is shown to be unique. The problem is also…
We introduce a new approach to develop stochastic optimization algorithms for a class of stochastic composite and possibly nonconvex optimization problems. The main idea is to combine two stochastic estimators to create a new hybrid one. We…
In this paper, we describe a new algorithm that approximates the extreme eigenvalue/eigenvector pairs of a symmetric matrix. The proposed algorithm can be viewed as an extension of the Jacobi eigenvalue method for symmetric matrices…
We revisit the classical dual ascent algorithm for minimization of convex functionals in the presence of linear constraints, and give convergence results which apply even for non-convex functionals. We describe limit points in terms of the…
Chandrasekaran, Parrilo and Willsky (2010) proposed a convex optimization problem to characterize graphical model selection in the presence of unobserved variables. This convex optimization problem aims to estimate an inverse covariance…
Recent efforts on solving inverse problems in imaging via deep neural networks use architectures inspired by a fixed number of iterations of an optimization method. The number of iterations is typically quite small due to difficulties in…
We develop and analyze stochastic approximation algorithms for solving nested compositional bi-level optimization problems. These problems involve a nested composition of $T$ potentially non-convex smooth functions in the upper-level, and a…
We compute approximate solutions to inverse problems for determining parameters in differential equation models with stochastic data on output quantities. The formulation of the problem and modeling framework define a solution as a…
An algorithm is proposed, analyzed, and tested experimentally for solving stochastic optimization problems in which the decision variables are constrained to satisfy equations defined by deterministic, smooth, and nonlinear functions. It is…
In this paper, we investigate numerical solutions for inverse singular value problems (for short, ISVPs) arising in various applications. Inspired by the methodologies employed for inverse eigenvalue problems, we propose a Cayley-free…
This note considers the multidimensional unstructured sparse recovery problems. Examples include Fourier inversion and sparse deconvolution. The eigenmatrix is a data-driven construction with desired approximate eigenvalues and eigenvectors…
In this study, an algorithm for computing the inverse of periodic k banded matrices, which are needed for solving the differential equations by using the finite differences, the solution of partial differential equations and the solution of…
In Part I of this paper, we introduced a two dimensional eigenvalue problem (2DEVP) of a matrix pair and investigated its fundamental theory such as existence, variational characterization and number of 2D-eigenvalues. In Part II, we…
Advanced optimization algorithms such as Newton method and AdaGrad benefit from second order derivative or second order statistics to achieve better descent directions and faster convergence rates. At their heart, such algorithms need to…
An extremely common bottleneck encountered in statistical learning algorithms is inversion of huge covariance matrices, examples being in evaluating Gaussian likelihoods for a large number of data points. We propose general parallel…