Related papers: Time integration of quantized tensor trains using …

Low-Rank Adaptation (LoRA) is widely used to efficiently adapt Transformers by adding trainable low-rank matrices to attention projections. While effective, these matrices are considered independent for each attention projection (Query,…

This paper introduces a novel computational approach termed the Reduced Augmentation Implicit Low-rank (RAIL) method by investigating two predominant research directions in low-rank solutions to time-dependent partial differential equations…

Dynamic Low Rank (DLR) methods are a promising way to reduce the computational cost and memory footprint of the high-dimensional thermal radiative transfer (TRT) equations. The TRT equations are a system of nonlinear PDEs that model the…

Tensor Networks (TNs) are a computational paradigm used for representing quantum many-body systems. Recent works have shown how TNs can also be applied to perform Machine Learning (ML) tasks, yielding comparable results to standard…

Test-time training (TTT) with KV binding as sequence modeling layer is commonly interpreted as a form of online meta-learning that memorizes a key-value mapping at test time. However, our analysis reveals multiple phenomena that contradict…

Learning neural fields has been an active topic in deep learning research, focusing, among other issues, on finding more compact and easy-to-fit representations. In this paper, we introduce a novel low-rank representation termed Tensor…

Multirate integration is an increasingly relevant tool that enables scientists to simulate multiphysics systems. Existing multirate methods are designed for equations whose fast and slow variables can be linearly separated using additive or…

A adapted tensor-structured GMRES method for the TT format is proposed and investigated. The Tensor Train (TT) approximation is a robust approach to high-dimensional problems. One class of problems is solution of a linear system. In this…

We introduce a fully discrete scheme to solve a class of high-dimensional Mean Field Games systems. Our approach couples semi-Lagrangian (SL) time discretizations with Tensor-Train (TT) decompositions to tame the curse of dimensionality. By…

Real-time recurrent learning (RTRL) for sequence-processing recurrent neural networks (RNNs) offers certain conceptual advantages over backpropagation through time (BPTT). RTRL requires neither caching past activations nor truncating…

Double Reinforcement Learning (DRL) enables efficient inference for policy values in nonparametric Markov decision processes (MDPs), but existing methods face two major obstacles: (1) they require stringent intertemporal overlap conditions…

Combination of low-tensor rank techniques and the Fast Fourier transform (FFT) based methods had turned out to be prominent in accelerating various statistical operations such as Kriging, computing conditional covariance, geostatistical…

We propose a high order adaptive-rank implicit integrators for stiff time-dependent PDEs, leveraging extended Krylov subspaces to efficiently and adaptively populate low-rank solution bases. This allows for the accurate representation of…

Low-rank tensor approximation techniques attempt to mitigate the overwhelming complexity of linear algebra tasks arising from high-dimensional applications. In this work, we study the low-rank approximability of solutions to linear systems…

Dynamical low-rank approximation allows for solving large-scale matrix differential equations (MDEs) with significantly fewer degrees of freedom and has been applied to a growing number of applications. However, most existing techniques…

In this article, we derive a semi-Lagrangian scheme for the solution of the Vlasov equation represented as a low-parametric tensor. Grid-based methods for the Vlasov equation have been shown to give accurate results but their use has mostly…

This study, for the first time, investigates the use of tensor trains (TTs) to represent high-dimensional unsteady flamelet progress variable (UFPV) manifolds in chemically reacting computational fluid dynamics (CFD). The UFPV framework…

This study explores the quantisation-aware training (QAT) on time series Transformer models. We propose a novel adaptive quantisation scheme that dynamically selects between symmetric and asymmetric schemes during the QAT phase. Our…

The dynamical low-rank (DLR) approximation is an efficient technique to approximate the solution to matrix differential equations. Recently, the DLR method was applied to radiation transport calculations to reduce memory requirements and…

Solving linear systems is at the foundation of many algorithms. Recently, quantum linear system algorithms (QLSAs) have attracted great attention since they converge to a solution exponentially faster than classical algorithms in terms of…