Related papers: On the choice of initial guesses for the Newton-Ra…
Newton-type solvers have been extensively employed for solving a variety of nonlinear system of algebraic equations. However, for some complex nonlinear system of algebraic equations, efficiently solving these systems remains a challenging…
First order optimization algorithms play a major role in large scale machine learning. A new class of methods, called adaptive algorithms, were recently introduced to adjust iteratively the learning rate for each coordinate. Despite great…
Algorithms for computing rational generating functions of solutions of one-dimensional difference equations are well-known and easy to implement. We propose an algorithm for computing rational generating functions of solutions of…
We estimate the initial conditions of a multivariable dynamical system from a scalar signal, using a modified Newton-Raphson method incorporating the time evolution. We can estimate initial conditions of periodic and chaotic systems and the…
Digital image correlation (DIC) is a widely used optical metrology for quantitative deformation measurement due to its non-contact, low-cost, highly precise feature. DIC relies on nonlinear optimization algorithm. Thus it is quite important…
The mathematical runtime analysis of evolutionary algorithms traditionally regards the time an algorithm needs to find a solution of a certain quality when initialized with a random population. In practical applications it may be possible…
We develop a randomized Newton's method for solving differential equations, based on a fully connected neural network discretization. In particular, the randomized Newton's method randomly chooses equations from the overdetermined nonlinear…
The importance of an adequate inner loop starting point (as opposed to a sufficient inner loop stopping rule) is discussed in the context of a numerical optimization algorithm consisting of nested primal-dual proximal-gradient iterations.…
Algorithms for computing equilibria, optima, and fixed points in nonconvex problems often depend sensitively on practitioner-chosen initial conditions. When uniqueness of a solution is of interest, a common heuristic is to run such…
This paper proposes an algorithm for solving structured optimization problems, which covers both the backward-backward and the Douglas-Rachford algorithms as special cases, and analyzes its convergence. The set of fixed points of the…
The notion of Boolean logic backpropagation was introduced to build neural networks with weights and activations being Boolean numbers. Most of computations can be done with Boolean logic instead of real arithmetic, both during training and…
The research area of algorithms with predictions has seen recent success showing how to incorporate machine learning into algorithm design to improve performance when the predictions are correct, while retaining worst-case guarantees when…
In this work, we present a novel algorithm design methodology that finds the optimal algorithm as a function of inequalities. Specifically, we restrict convergence analyses of algorithms to use a prespecified subset of inequalities, rather…
We develop adaptive discretization algorithms for locally optimal experimental design of nonlinear prediction models. With these algorithms, we refine and improve a pertinent state-of-the-art algorithm in various respects. We establish…
In Part I, we defined a LASSO condition number and developed an algorithm -- for computing support sets (feature selection) of the LASSO minimisation problem -- that runs in polynomial time in the number of variables and the logarithm of…
We consider the problem of minimizing a sum of $n$ functions over a convex parameter set $\mathcal{C} \subset \mathbb{R}^p$ where $n\gg p\gg 1$. In this regime, algorithms which utilize sub-sampling techniques are known to be effective. In…
The Newton-Raphson method is a fundamental root-finding technique with numerous applications in physics. In this study, we propose a parameterized variant of the Newton-Raphson method, inspired by principles from physics. Through analytical…
This article proposes new perspectives for developing derivative based numerical algorithms, supported by the introduction of a generalized derivative operators. It demonstrates that these operators have the potential to enhance and extend…
We aim to solve a topology optimization problem where the distribution of material in the design domain is represented by a density function. To obtain candidates for local minima, we want to solve the first order optimality system via…
Estimating the mixing density of a mixture distribution remains an interesting problem in statistics literature. Using a stochastic approximation method, Newton and Zhang (1999) introduced a fast recursive algorithm for estimating the…