Related papers: Lazy Kronecker Product
A new algorithm is developed allowing the Monte Carlo study of a 1 + 1 dimensional theory in real time. The main algorithmic development is to avoid the explicit calculation of the Jacobian matrix and its determinant in the update process.…
We consider the problem of universal dynamic regret minimization under exp-concave and smooth losses. We show that appropriately designed Strongly Adaptive algorithms achieve a dynamic regret of $\tilde O(d^2 n^{1/5} C_n^{2/5} \vee d^2)$,…
In this paper we present an inexact zeroth-order method suitable for the solution nonsmooth and nonconvex stochastic composite optimization problems, in which the objective is split into a real-valued Lipschitz continuous stochastic…
We propose a new risk-constrained reformulation of the standard Linear Quadratic Regulator (LQR) problem. Our framework is motivated by the fact that the classical (risk-neutral) LQR controller, although optimal in expectation, might be…
In this paper, we consider the generalised (higher order) Langevin equation for the purpose of simulated annealing and optimisation of nonconvex functions. Our approach modifies the underdamped Langevin equation by replacing the Brownian…
Task Arithmetic yields a modular, scalable way to adapt foundation models. Combining multiple task vectors, however, can lead to cross-task interference, causing representation drift and degraded performance. Representation drift…
We give a fully dynamic deterministic algorithm for maintaining a maximal matching of an $n$-vertex graph in $\tilde{O}(n^{8/9})$ amortized update time. This breaks the long-standing $\Omega(n)$-update-time barrier on dense graphs,…
We present a Las Vegas algorithm for dynamically maintaining a minimum spanning forest of an $n$-node graph undergoing edge insertions and deletions. Our algorithm guarantees an $O(n^{o(1)})$ worst-case update time with high probability.…
We propose an algorithm to impute and forecast a time series by transforming the observed time series into a matrix, utilizing matrix estimation to recover missing values and de-noise observed entries, and performing linear regression to…
In parallel and distributed environments, generational evolutionary algorithms often do not exploit the full potential of the computation system since they have to wait until the entire population is evaluated before starting selection…
In this work, we improved the analysis of the running time of SparseGPT [Frantar, Alistarh ICML 2023] from $O(d^{3})$ to $O(d^{\omega} + d^{2+a+o(1)} + d^{1+\omega(1,1,a)-a})$ for any $a \in [0, 1]$, where $\omega$ is the exponent of matrix…
The Cholesky QR algorithm is an efficient communication-minimizing algorithm for computing the QR factorization of a tall-skinny matrix. Unfortunately it has the inherent numerical instability and breakdown when the matrix is…
We conjecture that the inherent difference in generalisation between adaptive and non-adaptive gradient methods in deep learning stems from the increased estimation noise in the flattest directions of the true loss surface. We demonstrate…
This paper provides a general solution for the Kronecker product decomposition (KPD) of vectors, matrices, and hypermatrices. First, an algorithm, namely, monic decomposition algorithm (MDA), is reviewed. It consists of a set of projections…
We consider the problem of learning a latent $k$-vertex simplex $K\subset\mathbb{R}^d$, given access to $A\in\mathbb{R}^{d\times n}$, which can be viewed as a data matrix with $n$ points that are obtained by randomly perturbing latent…
There have been several algorithms designed to optimise matrix multiplication. From schoolbook method with complexity $O(n^3)$ to advanced tensor-based tools with time complexity $O(n^{2.3728639})$ (lowest possible bound achieved), a lot of…
We investigate Monte Carlo updating algorithms for simulating $SU(N)$ Yang-Mills fields on a single-site lattice, such as for the Twisted Eguchi-Kawai model (TEK). We show that performing only over-relaxation (OR) updates of the gauge links…
The Kronecker product is a key algorithm and is ubiquitous across the physical, biological, and computation social sciences. Thus considerations of optimal implementation are important. The need to have high performance and computational…
Projection maintenance is one of the core data structure tasks. Efficient data structures for projection maintenance have led to recent breakthroughs in many convex programming algorithms. In this work, we further extend this framework to…
We develop a novel primal-dual algorithm to solve a class of nonsmooth and nonlinear compositional convex minimization problems, which covers many existing and brand-new models as special cases. Our approach relies on a combination of a new…