Related papers: A new algorithm for complex non orthogonal joint d…
The main goal of this paper is to generalize Jacobi and Gauss-Seidel methods for solving non-square linear system. Towards this goal, we present iterative procedures to obtain an approximate solution for non-square linear system. We derive…
A scaled conjugate gradient method that accelerates existing adaptive methods utilizing stochastic gradients is proposed for solving nonconvex optimization problems with deep neural networks. It is shown theoretically that, whether with…
We propose a general random subspace framework for unconstrained nonconvex optimization problems that requires a weak probabilistic assumption on the subspace gradient, which we show to be satisfied by various random matrix ensembles, such…
We propose an online learning algorithm for a class of machine learning models under a separable stochastic approximation framework. The essence of our idea lies in the observation that certain parameters in the models are easier to…
We propose a stochastic variance-reduced cubic regularized Newton method for non-convex optimization. At the core of our algorithm is a novel semi-stochastic gradient along with a semi-stochastic Hessian, which are specifically designed for…
Recent work in the field of signal processing has shown that the singular value decomposition of a matrix with entries in certain real algebras can be a powerful tool. In this article we show how to generalise the QR decomposition and SVD…
We study the use of feedforward neural networks (FNN) to develop models of nonlinear dynamical systems from data. Emphasis is placed on predictions at long times, with limited data availability. Inspired by global stability analysis, and…
The nonlinear conjugate gradient methods are known to be an effective approach for standard unconstrained optimization problems especially for large-scale problems. This paper proposes a proximal nonlinear conjugate gradient method, which…
The problem of determining the configuration of points from partial distance information, known as the Euclidean Distance Geometry (EDG) problem, is fundamental to many tasks in the applied sciences. In this paper, we propose two algorithms…
For any given neural network architecture a permutation of weights and biases results in the same functional network. This implies that optimization algorithms used to `train' or `learn' the network are faced with a very large number (in…
This study investigates the geometric linearization of constraint Hamiltonian systems using the Jacobi metric and the Eisenhart lift. We establish a connection between linearization and maximally symmetric spacetimes, focusing on the…
We develop a framework which allows us to prove the essential general quasi-orthogonality for the non-symmetric Johnson-Nedelec finite element/boundary element coupling. General quasi-orthogonality was first proposed in [Axioms of…
Efficient layout of large-scale graphs remains a challenging problem: the force-directed and dimensionality reduction-based methods suffer from high overhead for graph distance and gradient computation. In this paper, we present a new graph…
An algorithm for the generation of non-uniform, locally-orthogonal staggered unstructured spheroidal grids is described. This technique is designed to generate very high-quality staggered Voronoi/Delaunay meshes appropriate for general…
In the lines of our approach in \cite{Ouorou2019}, where we exploit Nesterov fast gradient concept \cite{Nesterov1983} to the Moreau-Yosida regularization of a convex function, we devise new proximal algorithms for nonsmooth convex…
Several recent randomized linear algebra algorithms rely upon fast dimension reduction methods. A popular choice is the Subsampled Randomized Hadamard Transform (SRHT). In this article, we address the efficacy, in the Frobenius and spectral…
We present a new paradigm for speeding up randomized computations of several frequently used functions in machine learning. In particular, our paradigm can be applied for improving computations of kernels based on random embeddings. Above…
We exhibit a randomized algorithm which given a matrix $A\in \mathbb{C}^{n\times n}$ with $\|A\|\le 1$ and $\delta>0$, computes with high probability an invertible $V$ and diagonal $D$ such that $\|A-VDV^{-1}\|\le \delta$ using…
We are interested in numerically approximating the solution ${\bf U}(t)$ of the large dimensional semilinear matrix differential equation $\dot{\bf U}(t) = { \bf A}{\bf U}(t) + {\bf U}(t){ \bf B} + {\cal F}({\bf U},t)$, with appropriate…
Treewidth is a measure of how tree-like a graph is. It has many important algorithmic applications because many NP-hard problems on general graphs become tractable when restricted to graphs of bounded treewidth. Algorithms for problems on…