Related papers: Modelling Localisation and Spatial Scaling of Cons…

Positional encoding is essential for large language models (LLMs) to represent sequence order, yet recent studies show that Rotary Position Embedding (RoPE) can induce massive activation. We investigate the source of these instabilities via…

Autoregressive sequence modeling stands as the cornerstone of modern Generative AI, powering results across diverse modalities ranging from text generation to image generation. However, a fundamental limitation of this paradigm is the rigid…

The success of models operating on tokenized data has heightened the need for effective tokenization methods, particularly in vision and auditory tasks where inputs are naturally continuous. A common solution is to employ Vector…

In the present work, a machine learning based constitutive model for electro-mechanically coupled material behavior at finite deformations is proposed. Using different sets of invariants as inputs, an internal energy density is formulated…

Necking instabilities, in which tensile (extensional) deformation localizes into a small spatial region, are generic failure modes in elasto-viscoplastic materials. Materials in this very broad class --- including amorphous, crystalline,…

In the realm of large-scale language models, a significant challenge arises when extrapolating sequences beyond the maximum allowable length. This is because the model's position embedding mechanisms are limited to positions encountered…

Reinforcement Learning methods are capable of solving complex problems, but resulting policies might perform poorly in environments that are even slightly different. In robotics especially, training and deployment conditions often vary and…

This paper aims to investigate the numerical approximation of a general second order parabolic stochastic partial differential equation(SPDE) driven by multiplicative and additive noise. Our main interest is on such SPDEs where the…

In this paper, we present numerical simulations with local and nonlocal models under dynamic loading conditions. We show that for finite element (FE) computations of high-velocity, impact problems with softening material models will result…

Rotary Position Embedding (RoPE) has shown strong performance in text-based Large Language Models (LLMs), but extending it to video remains a challenge due to the intricate spatiotemporal structure of video frames. Existing adaptations,…

Discrete fine-scale models, in the form of either particle or lattice models, have been formulated successfully to simulate the behavior of quasi-brittle materials whose mechanical behavior is inherently connected to fracture processes…

Classically, the mechanical response of materials is described through constitutive models, often in the form of constrained ordinary differential equations. These models have a very limited number of parameters, yet, they are extremely…

Scaling recommendation models into large recommendation models has become one of the most widely discussed topics. Recent efforts focus on components beyond the scaling embedding dimension, as it is believed that scaling embedding may lead…

We study the impact of the wetting properties on the immiscible displacement of a viscous fluid in disordered porous media. We present a novel pore-scale model that captures wettability and dynamic effects, including the spatiotemporal…

Without positional information, attention-based Transformer neural networks are permutation-invariant. Absolute or relative positional embeddings are the most popular ways to feed Transformer models with positional information. Absolute…

With the advent of affordable depth sensors, 3D capture becomes more and more ubiquitous and already has made its way into commercial products. Yet, capturing the geometry or complete shapes of everyday objects using scanning devices (e.g.…

Recently, unsupervised constitutive model discovery has gained attention through frameworks based on the Virtual Fields Method (VFM), most prominently the EUCLID approach. However, the performance of VFM-based approaches, including EUCLID,…

The estimation of patient-specific tissue properties in the form of model parameters is important for personalized physiological models. However, these tissue properties are spatially varying across the underlying anatomical model,…

Localized deformation patterns are a common motif in morphogenesis and are increasingly finding widespread applications in materials science, for instance as memory devices. Here we describe the emergence of spatially localized deformations…

The choice of representation for geographic location significantly impacts the accuracy of models for a broad range of geospatial tasks, including fine-grained species classification, population density estimation, and biome classification.…