Related papers: In-memory eigenvector computation in time O(1)
Photonic neuromorphic computing may offer promising applications for a broad range of photonic sensors, including optical fiber sensors, to enhance their functionality while avoiding loss of information, energy consumption, and latency due…
The exponential speedups promised by Hamiltonian simulation on a quantum computer depends crucially on structure in both the Hamiltonian $\hat{H}$, and the quantum circuit $\hat{U}$ that encodes its description. In the quest to better…
We prove a universal limit theorem for the halting time, or iteration count, of the power/inverse power methods and the QR eigenvalue algorithm. Specifically, we analyze the required number of iterations to compute extreme eigenvalues of…
The Hessian matrix has a number of important applications in a variety of different fields, such as optimzation, image processing and statistics. In this paper we focus on the practical aspects of efficiently computing Hessian matrices in…
A new approach to solving eigenvalue optimization problems for large structured matrices is proposed and studied. The class of optimization problems considered is related to computing structured pseudospectra and their extremal points, and…
Quantum computing is a hotspot technology for its potential to accelerate specific applications by exploiting quantum parallelism. However, current physical quantum computers are limited to a relatively small scale, simulators based on…
Transformer networks, driven by self-attention, are central to Large Language Models. In generative Transformers, self-attention uses cache memory to store token projections, avoiding recomputation at each time step. However, GPU-stored…
Representations of quantum computations are almost always based on a tensor product $\otimes$-structure. This coincides with what we are able to execute in our experiments, as well as what we observe in Nature, but it makes certain familiar…
We present a novel class of methods to compute functions of matrices or their action on vectors that are suitable for parallel programming. Solving appropriate simple linear systems of equations in parallel (or computing the inverse of…
As CMOS scaling reaches its technological limits, a radical departure from traditional von Neumann systems, which involve separate processing and memory units, is needed in order to significantly extend the performance of today's computers.…
A new inverse iteration algorithm that can be used to compute all the eigenvectors of a real symmetric tri-diagonal matrix on parallel computers is developed. The modified Gram-Schmidt orthogonalization is used in the classical inverse…
In this paper, we first present an explicit expression for the inverse\emph{} of a type of matrices. As special applications, the inverse of some matrices arising from implicit time integration techniques, such as the well-known implicit…
Transformers are central to advances in artificial intelligence (AI), excelling in fields ranging from computer vision to natural language processing. Despite their success, their large parameter count and computational demands challenge…
Numerical Simulation is an essential part of the design and optimisation of astronomical adaptive optics systems. Simulations of adaptive optics are computationally expensive and the problem scales rapidly with telescope aperture size, as…
Recording reliably extracellular neural activities isan essential prerequisite for the development of bioelectronicsand neuroprosthetic applications. Recently, a fully differential,2-stage, integrating pre-amplifier was proposed for…
The transformer has revolutionized modern AI across language, vision, and beyond. It consists of $L$ layers, each running $H$ attention heads in parallel and feeding the combined output to the subsequent layer. In attention, the input…
In-memory computing is an emerging computing paradigm that overcomes the limitations of exiting Von-Neumann computing architectures such as the memory-wall bottleneck. In such paradigm, the computations are performed directly on the data…
We propose a new way of visualising the dynamics of iterative eigenvalue algorithms such as the QR algorithm, over the important special case of PSD (positive semi-definite) matrices. Many subtle and important properties of such algorithms…
In this paper, we consider the problem of efficiently computing the eigenvalues of limited-memory quasi-Newton matrices that exhibit a compact formulation. In addition, we produce a compact formula for quasi-Newton matrices generated by any…
In-memory computing with resistive crossbar arrays has been suggested to accelerate deep-learning workloads in highly efficient manner. To unleash the full potential of in-memory computing, it is desirable to accelerate the training as well…