Related papers: On Concentration Inequalities for Random Matrix Pr…
At the core of any inference procedure in deep neural networks are dot product operations, which are the component that require the highest computational resources. A common approach to reduce the cost of inference is to reduce its memory…
Attention mechanisms have revolutionized several domains of artificial intelligence, such as natural language processing and computer vision, by enabling models to selectively focus on relevant parts of the input data. While recent work has…
Let $\mathbf{A}$ be an $n\times n$-matrix over $\mathbb{F}_2$ whose every entry equals $1$ with probability $d/n$ independently for a fixed $d>0$. Draw a vector $\mathbf{y}$ randomly from the column space of $\mathbf{A}$. It is a simple…
We study infinite-horizon Discounted Markov Decision Processes (DMDPs) under a generative model. Motivated by the Algorithm with Advice framework Mitzenmacher and Vassilvitskii 2022, we propose a novel framework to investigate how a…
We consider filtering for a continuous-time, or asynchronous, stochastic system where the full distribution over states is too large to be stored or calculated. We assume that the rate matrix of the system can be compactly represented and…
We study the convergence properties of a pair of learning algorithms (learning with and without memory). This leads us to study the dominant eigenvalue of a class of random matrices. This turns out to be related to the roots of the…
Transformer models have achieved remarkable results in a wide range of applications. However, their scalability is hampered by the quadratic time and memory complexity of the self-attention mechanism concerning the sequence length. This…
The computation of a matrix function $f(A)$ is an important task in scientific computing appearing in machine learning, network analysis and the solution of partial differential equations. In this work, we use only matrix-vector products…
We study the rate of convergence of the empirical spectral distribution of products of independent non-Hermitian random matrices to the power of the Circular Law. The distance to the deterministic limit distribution will be measured in…
In this paper, we investigate a stochastic approximation procedure $\left(X_n\right)_{n\ge 0}$ taking values in $R$. The process is adapted to a filtration $(F_n)_{n\ge 0}$ and satisfies the recursion…
We address the following question: given a collection $\{\mathbf{A}^{(1)}, \dots, \mathbf{A}^{(N)}\}$ of independent $d \times d$ random matrices drawn from a common distribution $\mathbb{P}$, what is the probability that the centralizer of…
We use an extension of the diagrammatic rules in random matrix theory to evaluate spectral properties of finite and infinite products of large complex matrices and large hermitian matrices. The infinite product case allows us to define a…
We study the problem of efficiently correcting an erroneous product of two $n\times n$ matrices over a ring. Among other things, we provide a randomized algorithm for correcting a matrix product with at most $k$ erroneous entries running in…
Solving the generalized eigenvalue problem is a useful method for finding energy eigenstates of large quantum systems. It uses projection onto a set of basis states which are typically not orthogonal. One needs to invert a matrix whose…
1. A standard Gaussian random matrix has full rank with probability 1 and is well-conditioned with a probability quite close to 1 and converging to 1 fast as the matrix deviates from square shape and becomes more rectangular. 2. If we…
We mainly study numerical integration of real valued functions defined on the $d$-dimensional unit cube with all partial derivatives up to some finite order $r\ge1$ bounded by one. It is well known that optimal algorithms that use $n$…
Optimization in machine learning typically deals with the minimization of empirical objectives defined by training data. However, the ultimate goal of learning is to minimize the error on future data (test error), for which the training…
Matrices are exceptionally useful in various fields of study as they provide a convenient framework to organize and manipulate data in a structured manner. However, modern matrices can involve billions of elements, making their storage and…
We derive novel concentration inequalities for the operator norm of the sum of self-adjoint operators that do not explicitly depend on the underlying dimension of the operator, but rather an intrinsic notion of it. Our analysis leads to…
Randomized SVD has become an extremely successful approach for efficiently computing a low-rank approximation of matrices. In particular the paper by Halko, Martinsson, and Tropp (SIREV 2011) contains extensive analysis, and has made it a…