Related papers: The Unreasonable Effectiveness of Structured Rando…
Modern large language models (LLMs) excel at tasks that require storing and retrieving knowledge, such as factual recall and question answering. Transformers are central to this capability because they can encode information during training…
Tomographic imaging is useful for revealing the internal structure of a 3D sample. Classical reconstruction methods treat the object of interest as a vector to estimate its value. Such an approach, however, can be inefficient in analyzing…
As a randomized learner model, SCNs are remarkable that the random weights and biases are assigned employing a supervisory mechanism to ensure universal approximation and fast learning. However, the randomness makes SCNs more likely to…
Decentralized learning often involves a weighted global loss with heterogeneous node weights $\lambda$. We revisit two natural strategies for incorporating these weights: (i) embedding them into the local losses to retain a uniform weight…
We provide a self-contained introduction to random matrices. While some applications are mentioned, our main emphasis is on three different approaches to random matrix models: the Coulomb gas method and its interpretation in terms of…
We present an overview of randomized orthogonalization techniques that construct a well-conditioned basis whose sketch is orthonormal. Randomized orthogonalization has recently emerged as a powerful paradigm for reducing the computational…
Selecting a good column (or row) subset of massive data matrices has found many applications in data analysis and machine learning. We propose a new adaptive sampling algorithm that can be used to improve any relative-error column selection…
We consider the problem of recovering an unknown low-rank matrix X with (possibly) non-orthogonal, effectively sparse rank-1 decomposition from measurements y gathered in a linear measurement process A. We propose a variational formulation…
Many finance, physics, and engineering phenomena are modeled by continuous-time dynamical systems driven by highly irregular (stochastic) inputs. A powerful tool to perform time series analysis in this context is rooted in rough path theory…
Low-rank approximation of a matrix by means of structured random sampling has been consistently efficient in its extensive empirical studies around the globe, but adequate formal support for this empirical phenomenon has been missing so…
Randomized algorithms for low-rank approximation of quaternion matrices have gained increasing attention in recent years. However, existing methods overlook pass efficiency, the ability to limit the number of passes over the input…
We propose a scheme for recycling Gaussian random vectors into structured matrices to approximate various kernel functions in sublinear time via random embeddings. Our framework includes the Fastfood construction as a special case, but also…
In these lecture notes we present some connections between random matrices, the asymmetric exclusion process, random tilings. These three apparently unrelated objects have (sometimes) a similar mathematical structure, an interlacing…
The problem of learning long-term dependencies in sequences using Recurrent Neural Networks (RNNs) is still a major challenge. Recent methods have been suggested to solve this problem by constraining the transition matrix to be unitary…
The Johnson-Lindenstrauss (JL) theorem states that a set of points in high-dimensional space can be embedded into a lower-dimensional space while approximately preserving pairwise distances with high probability Johnson and Lindenstrauss…
Low rank approximation is an important tool used in many applications of signal processing and machine learning. Recently, randomized sketching algorithms were proposed to effectively construct low rank approximations and obtain approximate…
Let $\mathcal{M}$ be a smooth $d$-dimensional submanifold of $\mathbb{R}^N$ with boundary that's equipped with the Euclidean (chordal) metric, and choose $m \leq N$. In this paper we consider the probability that a random matrix $A \in…
In this paper we derive an efficient algorithm to learn the parameters of structured predictors in general graphical models. This algorithm blends the learning and inference tasks, which results in a significant speedup over traditional…
A particularly interesting instance of supervised learning with kernels is when each training example is associated with two objects, as in pairwise classification (Brunner et al., 2012), and in supervised learning of preference relations…
Orthogonal arrays are arguably one of the most fascinating and important statistical tools for efficient data collection. They have a simple, natural definition, desirable properties when used as fractional factorials, and a rich and…