Related papers: Simplified equations for the semi-localized transi…
In this paper we propose local approximation spaces for localized model order reduction procedures such as domain decomposition and multiscale methods. Those spaces are constructed from local solutions of the partial differential equation…
This paper deals with the exact calibration of semidiscretized stochastic local volatility (SLV) models to their underlying semidiscretized local volatility (LV) models. Under an SLV model, it is common to approximate the fair value of…
Model compression methods are used to reduce the computation and energy requirements for Large Language Models (LLMs). Quantization Aware Training (QAT), an effective model compression method, is proposed to reduce performance degradation…
This paper describes a localized algorithm for the topological simplification of scalar data, an essential pre-processing step of topological data analysis (TDA). Given a scalar field f and a selection of extrema to preserve, the proposed…
In the quest for next-generation sequence modeling architectures, State Space Models (SSMs) have emerged as a potent alternative to transformers, particularly for their computational efficiency and suitability for dynamical systems. This…
The QLP decomposition is one of the effective algorithms to approximate singular value decomposition (SVD) in numerical linear algebra. In this paper, we propose some single-pass randomized QLP decomposition algorithms for computing the…
We present a novel probabilistic deep learning approach, the 'Stochastic Latent Transformer' (SLT), designed for the efficient reduced-order modelling of stochastic partial differential equations. Stochastically driven flow models are…
Two moment-matching methods for model reduction of linear switched systems (LSSs) are presented. The methods are similar to the Krylov subspace methods used for moment matching for linear systems. The more general one of the two methods, is…
With the rise of Transformer models in NLP and CV domain, Multi-Head Attention has been proven to be a game-changer. However, its expensive computation poses challenges to the model throughput and efficiency, especially for the long…
We present a numerical investigation into the stochastic parameterisations of the Primitive Equations (PE) using the Stochastic Advection by Lie Transport (SALT) and Stochastic Forcing by Lie Transport (SFLT) frameworks. These frameworks…
As large language models (LLMs) become more prevalent, there is a growing need for new and improved quantization methods that can meet the computationalast layer demands of these modern architectures while maintaining the accuracy. In this…
The matrix quantization entails representing matrix elements in a more space-efficient form to reduce storage usage, with dequantization restoring the original matrix for use. We formulate the Quantization Error Minimization (QEM) problem…
LLM post-training typically propagates task gradients through the full depth of the model. Although this end-to-end structure is simple and general, it couples task adaptation to full-depth activation storage, long-range backward…
A multiscale numerical method is proposed for the solution of semi-linear elliptic stochastic partial differential equations with localized uncertainties and non-linearities, the uncertainties being modeled by a set of random parameters. It…
Semi-Lagrangian (SL) schemes are known as a major numerical tool for solving transport equations with many advantages and have been widely deployed in the fields of computational fluid dynamics, plasma physics modeling, numerical weather…
Machine learning potentials (MLP) have revolutionized the field of atomistic simulations by describing the atomic interactions with the accuracy of electronic structure methods at a small fraction of the costs. Most current MLPs construct…
Finite element based simulation of phenomena governed by partial differential equations is a standard tool in many engineering workflows today. However, the simulation of complex geometries is computationally expensive. Many engineering…
A growing trend has emerged in designing high-quality Small Language Models (SLMs) with a few million parameters. This trend is driven by the increasing concerns over cloud costs, privacy, and latency. Considering that full parameter…
Maximum simulated likelihood estimation of mixed multinomial logit (MMNL) or probit models requires evaluation of a multidimensional integral. Quasi-Monte Carlo (QMC) methods such as shuffled and scrambled Halton sequences and modified…
Lie-Trotter-Suzuki decompositions are an efficient way to approximate operator exponentials $\exp(t H)$ when $H$ is a sum of $n$ (non-commuting) terms which, individually, can be exponentiated easily. They are employed in time-evolution…