Related papers: The Legendre Transform in Modern Optimization

In this paper we introduce a geometric description of Lagrangian and Hamiltonian classical field theories on Lie algebroids in the framework of $k$-cosymplectic geometry. We discuss the relation between Lagrangian and Hamiltonian…

In large-eddy simulations (LES) a computational-domain translation velocity can be used to improve performance by allowing longer time-step intervals. The continuous equations are Galilean invariant, however, standard…

Lexical simplification (LS) aims to replace complex words in a given sentence with their simpler alternatives of equivalent meaning. Recently unsupervised lexical simplification approaches only rely on the complex word itself regardless of…

Lie group theory states that knowledge of a $m$-parameters solvable group of symmetries of a system of ordinary differential equations allows to reduce by $m$ the number of equation. We apply this principle by finding dilatations and…

Simulating the time evolution of Partial Differential Equations (PDEs) of large-scale systems is crucial in many scientific and engineering domains such as fluid dynamics, weather forecasting and their inverse optimization problems.…

As we know differential equations are very useful for electrical engineers to solve a variety of problems like: voltage across a capacitor, input versus output voltage, etc. Therefore, the goal of this paper is to find the solutions of…

We generalize quasi-arithmetic means beyond scalars by considering the gradient map of a Legendre type real-valued function. The gradient map of a Legendre type function is proven strictly comonotone with a global inverse. It thus yields a…

We present a framework for edge-aware optimization that is an order of magnitude faster than the state of the art while having comparable performance. Our key insight is that the optimization can be formulated by leveraging properties of…

Analytic expressions for the Fourier transforms of the Chebyshev and Legendre polynomials are derived, and the latter is used to find a new representation for the half-order Bessel functions. The numerical implementation of the so-called…

A systematic approach to finding variational approximation in an otherwise intractable non-conjugate model is to exploit the general principle of convex duality by minorizing the marginal likelihood that renders the problem tractable. While…

We show that the standard discrete update rule of transformer layers can be naturally interpreted as a forward Euler discretization of a continuous dynamical system. Our Transformer Flow Approximation Theorem demonstrates that, under…

High-order numerical methods enhance Transformer performance in tasks like NLP and CV, but introduce a performance-efficiency trade-off due to increased computational overhead. Our analysis reveals that conventional efficiency techniques,…

Gradient-based meta-learning techniques are both widely applicable and proficient at solving challenging few-shot learning and fast adaptation problems. However, they have practical difficulties when operating on high-dimensional parameter…

We consider linear optimization over a fixed compact convex feasible region that is semi-algebraic (or, more generally, "tame"). Generically, we prove that the optimal solution is unique and lies on a unique manifold, around which the…

Discrete analogs of the classical Mehler-Fock transforms are introduced and investigated. It involves series with the associated Legendre function $P^\mu_{in-1/2}(x), x > 1,\ {\rm Re} \mu < 1/2, \ n \in \mathbb{N}, i $ is the imaginary…

We investigate the relationship between finite terms in {\lambda}-letrec, the {\lambda}-calculus with letrec, and the infinite {\lambda}-terms they express. We say that a lambda-letrec term expresses a lambda-term if the latter can be…

Let $K$ denote a field and let $V$ denote a vector space over $K$ with finite positive dimension. We consider an ordered pair of linear transformations $A:V\to V$ and $A^*:V\to V$ that satisfy conditions (i), (ii) below. (i) There exists a…

We study learning to learn for regression problems through the lens of hyperparameter tuning. We propose the Langevin Gradient Descent Algorithm (LGD), which approximates the mean of the posterior distribution defined by the loss function…

In this paper, we propose a coupled tensor norm regularization that could enable the model output feature and the data input to lie in a low-dimensional manifold, which helps us to reduce overfitting. We show this regularization term is…

Linear Transformers and State Space Models have emerged as efficient alternatives to softmax Transformers for causal sequence modeling, enabling parallel training via matrix multiplication and efficient RNN-style inference. However, despite…