Related papers: Orthogonal Least Squares Algorithm for the Approxi…
The present study proposes incorporating non-parametric knowledge into the diffusion least-mean-squares algorithm in the framework of a maximum a posteriori (MAP) estimation. The proposed algorithm leads to a robust estimation of an unknown…
The following document presents some novel numerical methods valid for one and several variables, which using the fractional derivative, allow to find solutions for some non-linear systems in the complex space using real initial conditions.…
Recently, orthogonal recurrent neural networks (RNNs) have emerged as state-of-the-art models for learning long-term dependencies. This class of models mitigates the exploding and vanishing gradients problem by design. In this work, we…
A sequential quadratic optimization algorithm for minimizing an objective function defined by an expectation subject to nonlinear inequality and equality constraints is proposed, analyzed, and tested. The context of interest is when it is…
The Partition of Unity (PU) method, performed with local Radial Basis Function (RBF) approximants, has been proved to be an effective tool for solving large scattered data interpolation problems. However, in order to achieve a good…
Some response surface functions in complex engineering systems are usually highly nonlinear, unformed, and expensive-to-evaluate. To tackle this challenge, Bayesian optimization, which conducts sequential design via a posterior distribution…
We examine a wide class of stochastic approximation algorithms for solving (stochastic) nonlinear problems on Riemannian manifolds. Such algorithms arise naturally in the study of Riemannian optimization, game theory and optimal transport,…
We present a non-intrusive gradient and a non-intrusive BFGS algorithm for parameter estimation problems in non-stationary elasticity. To avoid multiple (and potentially expensive) solutions of the underlying partial differential equation…
This work presents study on regularized and non-regularized versions of perceptron learning and least squares algorithms for classification problems. Fr'echet derivatives for regularized least squares and perceptron learning algorithms are…
We present two new classes of orthogonal functions, log orthogonal functions (LOFs) and generalized log orthogonal functions (GLOFs), which are constructed by applying a $\log$ mapping to Laguerre polynomials. We develop basic approximation…
The partial least squares algorithm for dependent data realisations is considered. Consequences of ignoring the dependence for the algorithm performance are studied both theoretically and in simulations. It is shown that ignoring certain…
We introduce a novel approach to perform first-order optimization with orthogonal and unitary constraints. This approach is based on a parametrization stemming from Lie group theory through the exponential map. The parametrization…
Random feature (RF) method is a powerful kernel approximation technique, but is typically equipped with fixed activation functions, limiting its adaptability across diverse tasks. To overcome this limitation, we introduce the Random Feature…
This paper analyzes local convergence of the block Newton (BN) method introduced in [5, 6] for one-dimensional shallow neural network approximation to functions and diffusion-reaction problems. The BN method consists of the 2x2 block…
We present a comprehensive study of radial basis function (RBF) approximations for elliptic and obstacle-type boundary value problems under a variational formulation. Our focus is on practical accuracy, robustness and efficiency. To address…
In this paper, we study the equality constrained nonlinear least squares problem, where the Jacobian matrices of the objective function and constraints are unavailable or expensive to compute. We approximate the Jacobian matrices via…
Radial basis function generated finite-difference (RBF-FD) methods have recently gained popularity due to their flexibility with irregular node distributions. However, the convergence theories in the literature, when applied to nonuniform…
We develop a corrective mechanism for neural network approximation: the total available non-linear units are divided into multiple groups and the first group approximates the function under consideration, the second group approximates the…
Kernel methods are widespread in machine learning; however, they are limited by the quadratic complexity of the construction, application, and storage of kernel matrices. Low-rank matrix approximation algorithms are widely used to address…
Let $f\in L^1(\R^d)$ be real. The Rudin-Osher-Fatemi model is to minimize $\|u\|_{\dot{BV}}+\lambda\|f-u\|_{L^2}^2$, in which one thinks of $f$ as a given image, $\lambda > 0$ as a "tuning parameter", $u$ as an optimal "cartoon"…