Related papers: HMC and gradient flow with machine-learned classic…
This paper tackles the unconstrained minimization of a class of nonsmooth and nonconvex functions that can be written as finite max-functions. A gradient and function-based sampling method is proposed which, under special circumstances,…
We show that an infinitesimal step of gradient flow can be used for defining a novel approach for computing gradients of physical observables with respect to action parameters. Compared to the commonly used perturbative expansion, this…
We study gradient flow on the multi-index regression problem for high-dimensional Gaussian data. Multi-index functions consist of a composition of an unknown low-rank linear projection and an arbitrary unknown, low-dimensional link…
Recent advances in MCMC use normalizing flows to precondition target distributions and enable jumps to distant regions. However, there is currently no systematic comparison of different normalizing flow architectures for MCMC. As such, many…
Flow matching models have shown great potential in image generation tasks among probabilistic generative models. However, most flow matching models in the literature do not explicitly utilize the underlying clustering structure in the…
Action-constrained reinforcement learning (ACRL) is a popular approach for solving safety-critical and resource-allocation related decision making problems. A major challenge in ACRL is to ensure agent taking a valid action satisfying…
The move from hand-designed to learned optimizers in machine learning has been quite successful for gradient-based and -free optimizers. When facing a constrained problem, however, maintaining feasibility typically requires a projection…
We provide gradient flow interpretations for the continuous-time continuous-state Hopfield neural network (HNN). The ordinary and stochastic differential equations associated with the HNN were introduced in the literature as analog…
We adapt the precise definition of the flowing effective action in order to obtain a functional flow equation with simple properties close to physical intuition. The simplified flow equation is invariant under local gauge transformations…
A new parametrization is introduced for the fixed point (FP) action in SU(3) gauge theory using fat links. We investigate its scaling properties by means of the static quark-antiquark potential and the dimensionless quantities $r_0 T_c,…
Normalizing Flows (NF) are powerful generative models with increasing applications in augmenting Monte Carlo algorithms due to their high flexibility and expressiveness. In this work we explore the integration of NF in Diagrammatic Monte…
Hamiltonian Monte Carlo is a widely used algorithm for sampling from posterior distributions of complex Bayesian models. It can efficiently explore high-dimensional parameter spaces guided by simulated Hamiltonian flows. However, the…
(Neal and Hinton, 1998) recast maximum likelihood estimation of any given latent variable model as the minimization of a free energy functional $F$, and the EM algorithm as coordinate descent applied to $F$. Here, we explore alternative…
Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC) approach that exhibits favourable exploration properties in high-dimensional models such as neural networks. Unfortunately, HMC has limited use in large-data regimes and…
Motivated by the emerging role of interpolating machines in signal processing and machine learning, this work considers the computational aspects of over-parametrized matrix factorization. In this context, the optimization landscape may…
We study fractional configurations in gravity theories and Lagrange mechanics. The approach is based on Caputo fractional derivative which gives zero for actions on constants. We elaborate fractional geometric models of physical…
We report developments of the kinetic Monte Carlo (KMC) method with improved accuracy and increased versatility for the description of atomic diffusivity on metal surfaces. The on-lattice constraint built into our recently proposed…
Gradient clipping is an important technique for deep neural networks with exploding gradients, such as recurrent neural networks. Recent studies have shown that the loss functions of these networks do not satisfy the conventional smoothness…
Wasserstein gradient flows on probability measures have found a host of applications in various optimization problems. They typically arise as the continuum limit of exchangeable particle systems evolving by some mean-field interaction…
We study the dynamics of gradient flow for training a multi-head softmax attention model for in-context learning of multi-task linear regression. We establish the global convergence of gradient flow under suitable choices of initialization.…