相关论文: MuCon: Clipped Muon Updates for LLM Training

Recent work in machine learning community proposed multiple methods for performing lossy compression (quantization) of large matrices. This quantization is important for accelerating matrix multiplication (main component of large language…

Submodular maximization is a general optimization problem with a wide range of applications in machine learning (e.g., active learning, clustering, and feature selection). In large-scale optimization, the parallel running time of an…

We consider three-loop radiative-recoil corrections to hyperfine splitting in muonium generated by the diagrams with electron and muon vacuum polarizations. We calculate single-logarithmic and nonlogarithmic contributions of order…

A matrix optimization problem over an uncertain linear system on finite horizon (abbreviated as MOPUL) is studied, in which the uncertain transition matrix is regarded as a decision variable. This problem is in general NP-hard. By using the…

Knowledge of atmospheric muon flux intensity at large depths is extremely important for neutrino telescopes located deep under ground, water or ice. One of the methods to transform muon sea-level spectrum into depth one is to apply Monte…

We study mutually unbiased bases (MUBs) as structured finite initialization and adaptation families for variational quantum algorithms. The main theoretical result is that, in every dimension admitting a complete set of MUBs, the complete…

Several variations of adaptive first-order and second-order optimization methods have been proposed to accelerate and scale the training of large language models. The performance of these optimization routines is highly sensitive to the…

In the framework of a recently proposed extension of the Standard Model, with N=1 SuperSymmetry in 5 dimensions, compactified on R^1/(Z_2 x Z'_2), we compute the muon anomalous magnetic moment at one loop to order (M_W R)^2, where R is the…

We examine the \emph{submodular maximum coverage problem} (SMCP), which is related to a wide range of applications. We provide the first variational approximation for this problem based on the Nemhauser divergence, and show that it can be…

We consider the factorization of a rectangular matrix $X $ into a positive linear combination of rank-one factors of the form $u v^\top$, where $u$ and $v$ belongs to certain sets $\mathcal{U}$ and $\mathcal{V}$, that may encode specific…

Scaling large models requires optimization strategies that ensure rapid convergence grounded in stability. Maximal Update Parametrization ($\boldsymbol{\mu}$P) provides a theoretical safeguard for width-invariant $\Theta(1)$ activation…

Submodularity is one of the most well-studied properties of problem classes in combinatorial optimization and many applications of machine learning and data mining, with strong implications for guaranteed optimization. In this thesis, we…

Matrix-monotonic optimization exploits the monotonic nature of positive semi-definite matrices to derive optimal diagonalizable structures for the matrix variables of matrix-variable optimization problems. Based on the optimal structures…

Piecewise regression is a versatile approach used in various disciplines to approximate complex functions from limited, potentially noisy data points. In control, piecewise regression is, e.g., used to approximate the optimal control law of…

A reliable evaluation of the integral giving the hadronic vacuum polarization contribution to the muon anomalous magnetic moment should be possible using a simple trapezoid-rule integration of lattice data for the subtracted electromagnetic…

The training of foundational machine learning interatomic potentials (fMLIPs) relies on diverse databases with energies and forces calculated using ab initio methods. We show that fMLIPs trained on large datasets such as MPtrj, Alexandria,…

A novel conformal mapping based Fractional Order (FO) methodology is developed in this paper for tuning existing classical (Integer Order) Proportional Integral Derivative (PID) controllers especially for sluggish and oscillatory second…

Multi-Task Learning (MTL) is designed to train multiple correlated tasks simultaneously, thereby enhancing the performance of individual tasks. Typically, a multi-task network structure consists of a shared backbone and task-specific…

Prompt tuning and adapter tuning have shown great potential in transferring pre-trained vision-language models (VLMs) to various downstream tasks. In this work, we design a new type of tuning method, termed as regularized mask tuning, which…

In this paper we consider a general matrix factorization model which covers a large class of existing models with many applications in areas such as machine learning and imaging sciences. To solve this possibly nonconvex, nonsmooth and…