Related papers: Stable rank-adaptive Dynamically Orthogonal Runge-…
In [1] is proposed a simplified DeC method, that, when combined with the residual distribution (RD) framework, allows to construct a high order, explicit FE scheme with continuous approximation avoiding the inversion of the mass matrix for…
Domain-specific finetuning is essential for dense retrievers, yet not all training pairs contribute equally to the learning process. We introduce OPERA, a data pruning framework that exploits this heterogeneity to improve both the…
Direct collocation is a widely used method for solving dynamic optimization problems (DOPs), but its implementation simplicity and computational efficiency are limited for challenging problems like those involving singular arcs. In this…
When implementing the gradient descent method in low precision, the employment of stochastic rounding schemes helps to prevent stagnation of convergence caused by the vanishing gradient effect. Unbiased stochastic rounding yields zero bias…
In this paper, we apply the Paired-Explicit Runge-Kutta (P-ERK) schemes by Vermeire et. al. (2019, 2022) to dynamically partitioned systems arising from adaptive mesh refinement. The P-ERK schemes enable multirate time-integration with no…
Low-rank training methods reduce the number of trainable parameters by re-parameterizing the weights with matrix decompositions (e.g., singular value decomposition). However, enforcing a fixed low-rank structure caps the rank of the weight…
We present a second-order strictly length-preserving and unconditionally energy-stable rotational discrete gradient (Rdg) scheme for the numerical approximation of the Oseen-Frank gradient flows with anisotropic elastic energy functional.…
Low-rank gradient compression reduces communication in distributed training by representing updates with rank-$r$ factors. Dion is a recent method that approximates Muon, a spectral optimizer that orthogonalizes momentum, using one step of…
Convenient, easy to implement stochastic integration methods are developed on the basis of abstract one-step deterministic order $p$ integration techniques. The abstraction as an arbitrary one step map allows the inspection of easy to…
Implicit Runge--Kutta (IRK) methods are highly effective for solving stiff ordinary differential equations (ODEs) but can be computationally expensive for large-scale problems due to the need of solving coupled algebraic equations at each…
This work proposes an adaptive structure-preserving model order reduction method for finite-dimensional parametrized Hamiltonian systems modeling non-dissipative phenomena. To overcome the slowly decaying Kolmogorov width typical of…
In this master thesis we have compared different second order stabilized explicit Runge-Kutta methods when applied to the incompressible Navier-Stokes equations by means of a projection method and a differential algebraic approach. We…
Gradient descent with momentum has been widely applied in various signal processing and machine learning tasks, demonstrating a notable empirical advantage over standard gradient descent. However, momentum-based distributed Riemannian…
In this paper, we propose a domain decomposition dynamical low-rank method to solve high-dimensional radiative transfer problems and similar kinetic equations. The algorithm uses a separate low-rank approximation on each spatial subdomain,…
Stochastic gradient descent (SGD) provides a simple and efficient way to solve a broad range of machine learning problems. Here, we focus on distribution regression (DR), involving two stages of sampling: Firstly, we regress from…
Many core problems in robotics can be framed as constrained optimization problems. Often on these problems, the robotic system has uncertainty, or it would be advantageous to identify multiple high quality feasible solutions. To enable…
We study off-dynamics Reinforcement Learning (RL), where the policy training and deployment environments are different. To deal with this environmental perturbation, we focus on learning policies robust to uncertainties in transition…
We study Runge-Kutta methods for rough differential equations which can be used to calculate solutions to stochastic differential equations driven by processes that are rougher than a Brownian motion. We use a Taylor series representation…
Low-Rank Adaptation (LoRA) has proven effective in reducing computational costs while maintaining performance comparable to fully fine-tuned foundation models across various tasks. However, its fixed low-rank structure restricts its…
In distributed optimization problems, a technique called gradient coding, which involves replicating data points, has been used to mitigate the effect of straggling machines. Recent work has studied approximate gradient coding, which…