Related papers: A data scalable augmented Lagrangian KKT precondit…
This paper studies bilevel polynomial optimization problems. To solve them, we give a method based on polynomial optimization relaxations. Each relaxation is obtained from the Kurash-Kuhn-Tucker (KKT) conditions for the lower level…
The Hierarchical Kernel Transformer (HKT) is a multi-scale attention mechanism that processes sequences at L resolution levels via trainable causal downsampling, combining level-specific score matrices through learned convex weights. The…
Recent efforts to accelerate LLM pretraining have focused on computationally-efficient approximations that exploit second-order structure. This raises a key question for large-scale training: how much performance is forfeited by these…
Distribution network reconfiguration (DNR) is an effective approach for optimizing distribution network operation. However, the DNR problem is computationally challenging due to the mixed-integer non-convex nature. One feasible approach for…
The relaxed physical factorization (RPF) preconditioner is a recent algorithm allowing for the efficient and robust solution to the block linear systems arising from the three-field displacement-velocity-pressure formulation of coupled…
In this paper, we study second-order necessary and sufficient optimality conditions of Karush--Kuhn--Tucker-type for locally optimal solutions in the sense of Pareto to a class of multi-objective optimal control problems with mixed…
In this paper, we further investigate and refine the subspace-constrained preconditioning technique to enhance the theoretical and numerical convergence properties of randomized iterative methods for solving linear systems. In particular,…
This paper aims to provide various applications for second-order variational analysis of extended-real-valued piecewise liner functions recently obtained in [1]. We mainly focus here on establishing relationships between full stability of…
We develop a homotopy-based framework for computing Karush-Kuhn-Tucker (KKT) points of multiobjective optimization problems. The proposed homotopy map continuously deforms an easily solvable system into the KKT conditions associated with…
A forward model of matter and biased tracers at arbitrary order in Lagrangian perturbation theory (LPT) is presented. The forward model contains the complete LPT displacement field at any given order in perturbations, as well as all…
This work aims to accelerate the convergence of proximal gradient methods used to solve regularized linear inverse problems. This is achieved by designing a polynomial-based preconditioner that targets the eigenvalue spectrum of the normal…
Knowledge tracing (KT) is the problem of predicting students' future performance based on their historical interaction sequences. With the advanced capability of capturing contextual long-term dependency, attention mechanism becomes one of…
Rational Krylov subspace (RKS) techniques are well-established and powerful tools for projection-based model reduction of time-invariant dynamic systems. For hyperbolic wavefield problems, such techniques perform well in configurations…
In several geophysical applications, such as full waveform inversion and data modelling, we are facing the solution of inhomogeneous Helmholtz equation. The difficulties of solving the Helmholtz equa- tion are two fold. Firstly, in the case…
Bilevel optimization provides a powerful framework for modelling hierarchical decision-making systems. This work presents a sensitivity-based algorithm that addresses the bilevel structure directly by treating the lower-level optimal…
Augmented Lagrangian preconditioners have successfully yielded Reynolds-robust preconditioners for the stationary incompressible Navier-Stokes equations, but only for specific discretizations. The discretizations for which these…
This paper introduces a novel variational Bayesian method that integrates Tucker decomposition for efficient high-dimensional inverse problem solving. The method reduces computational complexity by transforming variational inference from a…
This paper pursues a two-fold goal. Firstly, we aim to derive novel second-order characterizations of important robust stability properties of perturbed Karush-Kuhn-Tucker systems for a broadclass of constrained optimization problems…
First-order optimization solvers, such as the Fast Gradient Method, are increasingly being used to solve Model Predictive Control problems in resource-constrained environments. Unfortunately, the convergence rate of these solvers is…
Computing the regularized solution of Bayesian linear inverse problems as well as the corresponding regularization parameter is highly desirable in many applications. This paper proposes a novel iterative method, termed the Projected Newton…