Related papers: The Legendre Transform in Modern Optimization

Learning-to-optimize (L2O) is an emerging research area in large-scale optimization with applications in data science. Recently, researchers have proposed a novel L2O framework called learned mirror descent (LMD), based on the classical…

Signature, lying at the heart of rough path theory, is a central tool for analysing controlled differential equations driven by irregular paths. Recently it has also found extensive applications in machine learning and data science as a…

In three space dimensions, when a physical system possesses spherical symmetry, the dynamical equations automatically lead to the Legendre and the associated Legendre equations, with the respective orthogonal polynomials as their standard…

Deep convolutional networks (convnets) show a remarkable ability to learn disentangled representations. In recent years, the generalization of deep learning to Lie groups beyond rigid motion in $\mathbb{R}^n$ has allowed to build convnets…

This paper revisits the special type of a neural network known under two names. In the statistics and machine learning community it is known as a multi-class logistic regression neural network. In the neural network community, it is simply…

A new method is described for solving optimal control problems using direct collocation at Legendre-Gauss-Lobatto points. The approach of this paper employs a polynomial approximation of the right-hand side vector field of the differential…

In this technical report, we analyze Legendre decomposition for non-negative tensor in theory and application. In theory, the properties of dual parameters and dually flat manifold in Legendre decomposition are reviewed, and the process of…

We show that the Laplace approximation of a supremum by Lp-norms has interesting consequences in optimization. For instance, the logarithmic barrier functions (LBF) of a primal convex problem P and its dual appear naturally when using this…

Gradient descent (GD) is known to converge quickly for convex objective functions, but it can be trapped at local minima. On the other hand, Langevin dynamics (LD) can explore the state space and find global minima, but in order to give…

Under conventional Legendre transformation, systems with a non-convex Lagrangian will result in a multi-valued Hamiltonian as a function of conjugate momentum. This causes problems such as non-unitary time evolution of quantum state and…

We study when the \emph{optimization curve} of first-order methods -- the sequence \${f(x\_n)}*{n\ge0}\$ produced by constant-stepsize iterations -- is convex, equivalently when the forward differences \$f(x\_n)-f(x*{n+1})\$ are…

Evolutionary algorithms serve as a powerful paradigm for tackling optimization challenges, yet their reliance on manually engineered heuristics inherently limits their adaptability across diverse landscapes. However, the transition from the…

The Laplace transform is a useful and powerful analytic tool with applications to several areas of applied mathematics, including differential equations, probability and statistics. Similarly to the inversion of the Fourier transform,…

We present a new class of equivariant neural networks, hereby dubbed Lattice-Equivariant Neural Networks (LENNs), designed to satisfy local symmetries of a lattice structure. Our approach develops within a recently introduced framework…

Convexity is an important notion in non linear optimization theory as well as in infinite dimensional functional analysis. As will be seen below, very simple and powerful tools will be derived from elementary duality arguments (which are…

Transform Invariant Low-rank Textures (TILT) is a novel and powerful tool that can effectively rectify a rich class of low-rank textures in 3D scenes from 2D images despite significant deformation and corruption. The existing algorithm for…

We review studies on the application of Lie group methods to delay ordinary differential equations (DODEs). For first- and second-order DODEs with a single delay parameter that depends on independent and dependent variables, the group…

We reveal the relation between the Legendre transform of convex functions and heat flow evolution, and how it applies to the functional Blaschke-Santalo inequality. We also describe local maximizers in this inequality.

Fix an algebraically closed field $\mathbb{F}$ and an integer $d \geq 3$. Let $V$ be a vector space over $\mathbb{F}$ with dimension $d+1$. A Leonard pair on $V$ is an ordered pair of diagonalizable linear transformations $A: V \to V$ and…

Here I investigate some mathematical aspects of the maximum entropy theory of ecology (METE). In particular I address the geometrical structure of METE endowed by information geometry. As novel results, the macrostate entropy is calculated…