Related papers: Generating Hadamard matrices with transformers
Randomized sampling has recently been proven a highly efficient technique for computing approximate factorizations of matrices that have low numerical rank. This paper describes an extension of such techniques to a wider class of matrices…
Due to the rapid growth of data and computational resources, distributed optimization has become an active research area in recent years. While first-order methods seem to dominate the field, second-order methods are nevertheless attractive…
We introduce Hadamard matrices whose entries are quaternionic. We then go on to provide classification of quaternionic Hadamard matrices of circulant core of orders 2 through 5. We also introduce quaternionic Hadamard matrices of Butson…
Hashing plays an important role in information retrieval, due to its low storage and high speed of processing. Among the techniques available in the literature, multi-modal hashing, which can encode heterogeneous multi-modal features into…
Search is an important technique in program synthesis that allows for adaptive strategies such as focusing on particular search directions based on execution results. Several prior works have demonstrated that neural models are effective at…
We explore efficient neural architecture search methods and show that a simple yet powerful evolutionary algorithm can discover new architectures with excellent performance. Our approach combines a novel hierarchical genetic representation…
We rationalize the somewhat surprising efficacy of the Hadamard transform in simplifying the eigenstates of the quantum baker's map, a paradigmatic model of quantum chaos. This allows us to construct closely related, but new, transforms…
Differentiable architecture search (DARTS) is successfully applied in many vision tasks. However, directly using DARTS for Transformers is memory-intensive, which renders the search process infeasible. To this end, we propose a multi-split…
In this paper, we present a stable and efficient approach for constructing Laguerre pseudospectral differentiation matrices. The proposed method reformulates the off-diagonal entries and computes all required quantities simultaneously using…
Identifying discrete patterns in binary data is an important dimensionality reduction tool in machine learning and data mining. In this paper, we consider the problem of low-rank binary matrix factorisation (BMF) under Boolean arithmetic.…
Random projections or sketching are widely used in many algorithmic and learning contexts. Here we study the performance of iterative Hessian sketch for least-squares problems. By leveraging and extending recent results from random matrix…
The Hadamard Extension of a matrix is the matrix consisting of all Hadamard products of subsets of its rows. This construction arises in the context of identifying a mixture of product distributions on binary random variables: full column…
In this paper, a class of deterministic sensing matrices are constructed by selecting rows from Fourier matrices. These matrices have better performance in sparse recovery than random partial Fourier matrices. The coherence and restricted…
Generating irreducible site-occupancy configurations by taking advantage of crystal symmetry is a ubiquitous method for accelerating of disordered structure prediction, which plays an important role in condensed matter physics and material…
While research in robust optimization has attracted considerable interest over the last decades, its algorithmic development has been hindered by several factors. One of them is a missing set of benchmark instances that make algorithm…
We introduce a construction that, given a pair (u,v) of complex Hadamard matrices of the same order, generates infinitely many biunitary matrices of varying (and distinct) orders. As a key application, this framework yields nested sequences…
In this paper we describe an approach to construct large extendable collections of vectors in predefined spaces of given dimensions. These collections are useful for neural network latent space configuration and training. For classification…
Quantizing the activation, weight, and gradient to 4-bit is promising to accelerate neural network training. However, existing 4-bit training methods require custom numerical formats which are not supported by contemporary hardware. In this…
A finite sequence of numbers is perfect if it has zero periodic autocorrelation after a nontrivial cyclic shift. In this work, we study quaternionic perfect sequences having a one-to-one correspondence with the binary sequences arising in…
The Hadamard decomposition is a powerful technique for data analysis and matrix compression, which decomposes a given matrix into the element-wise product of two or more low-rank matrices. In this paper, we develop an efficient algorithm to…