Related papers: Second order optimality on orthogonal Stiefel mani…
In this work, we address the problem of Hessian inversion bias in distributed second-order optimization algorithms. We introduce a novel shrinkage-based estimator for the resolvent of gram matrices which is asymptotically unbiased, and…
"Classical" First Order (FO) algorithms of convex optimization, such as Mirror Descent algorithm or Nesterov's optimal algorithm of smooth convex optimization, are well known to have optimal (theoretical) complexity estimates which do not…
Using the embedded gradient vector field method we explicitly compute the list of critical points of the free energy for a Cosserat body model. We also formulate necessary and sufficient conditions for critical points in the abstract case…
In the present work we studied a subfield of Applied Mathematics called Riemannian Optimization. The main goal of this subfield is to generalize algorithms, theorems and tools from Mathematical Optimization to the case in which the…
The optimal value function is one of the basic objects in the field of mathematical optimization, as it allows the evaluation of the variations in the cost/revenue generated while minimizing/maximizing a given function under some…
Hedging exotic options in presence of market frictions is an important risk management task. Deep hedging can solve such hedging problems by training neural network policies in realistic simulated markets. Training these neural networks may…
This work is on constrained large-scale non-convex optimization where the constraint set implies a manifold structure. Solving such problems is important in a multitude of fundamental machine learning tasks. Recent advances on Riemannian…
This paper introduces a novel Homogeneous Second-order Descent Ascent (HSDA) algorithm for nonconvex-strongly concave minimax optimization problems. At each iteration, HSDA uniquely computes a search direction by solving a homogenized…
Optimization problems on the generalized Stiefel manifold (and products of it) are prevalent across science and engineering. For example, in computational science they arise in symmetric (generalized) eigenvalue problems, in nonlinear…
This paper focus on the minimization of a possibly nonsmooth objective function over the Stiefel manifold. The existing approaches either lack efficiency or can only tackle prox-friendly objective functions. We propose a constraint…
An algorithm is proposed for solving optimization problems arising in neural network training for supervised learning. The unique feature of the algorithm is the use of an auxiliary loss, in addition to the original loss employed for model…
Stochastic gradient descent is a simple approach to find the local minima of a cost function whose evaluations are corrupted by noise. In this paper, we develop a procedure extending stochastic gradient descent algorithms to the case where…
This report investigates the fitting of the Hessian or its inverse for stochastic optimizations using a Hessian fitting criterion derived from the preconditioned stochastic gradient descent (PSGD) method. This criterion is closely related…
Optimization with orthogonality constraints frequently arises in various fields such as machine learning. Riemannian optimization offers a powerful framework for solving these problems by equipping the constraint set with a Riemannian…
Verifying the Second-Order Sufficient Condition (SOSC), thus ensuring a stationary point locally minimizes a given objective function (subject to certain constraints), is an essential component of non-convex computational optimization and…
In this paper, we investigate the multi-objective optimal control problem of ordinary differential equations on Riemannian manifolds. We first obtain the second-order necessary conditions for weak Pareto optimal solutions for…
This paper focuses on second-order necessary optimality conditions for constrained optimization problems on Banach spaces. For problems in the classical setting, where the objective function is $C^2$-smooth, we show that strengthened…
In this paper, we study stochastic non-convex optimization with non-convex random functions. Recent studies on non-convex optimization revolve around establishing second-order convergence, i.e., converging to a nearly second-order optimal…
In recent papers the author introduced a simple alternative to isoparametric finite elements of the n-simplex type, to enhance the accuracy of approximations of second-order boundary value problems with Dirichlet conditions, posed in smooth…
Manifold optimization is ubiquitous in computational and applied mathematics, statistics, engineering, machine learning, physics, chemistry and etc. One of the main challenges usually is the non-convexity of the manifold constraints. By…