Related papers: Softmax as a Lagrangian-Legendrian Seam

In state-of-the-art deep learning for object recognition, SoftMax and Sigmoid functions are most commonly employed as the predictor outputs. Such layers often produce overconfident predictions rather than proper probabilistic scores, which…

Differentiable physics simulation provides an avenue to tackle previously intractable challenges through gradient-based optimization, thereby greatly improving the efficiency of solving robotics-related problems. To apply differentiable…

We present a new method for learning Soft Random Geometric Graphs (SRGGs), drawn in probabilistic metric spaces, with the connection function of the graph defined as the marginal posterior probability of an edge random variable, given the…

The Softmax function is used in the final layer of nearly all existing sequence-to-sequence models for language generation. However, it is usually the slowest layer to compute which limits the vocabulary size to a subset of most frequent…

The Softmax function on top of a final linear layer is the de facto method to output probability distributions in neural networks. In many applications such as language models or text generation, this model has to produce distributions over…

The softmax representation of probabilities for categorical variables plays a prominent role in modern machine learning with numerous applications in areas such as large scale classification, neural language modeling and recommendation…

Soft lubricated contacts exhibit complex interfacial behaviours governed by the coupled effects of multiscale surface roughness and non-linear fluid-solid interactions. Accurately capturing this interplay across thin-film flows is…

The advantage of particle Lagrangian methods in computational fluid dynamics is that advection is accurately modeled. However, this complicates the calculation of space derivatives. If a mesh is employed, it must be updated at each time…

To help evaluate and understand the latent capabilities of language models, this paper introduces an approach using optimized input embeddings, or 'soft prompts,' as a metric of conditional distance between a model and a target behavior.…

In this paper we propose and test the validity of simple and easy-to-implement algorithms within the immersed boundary framework geared towards large scale simulations involving thousands of deformable bodies in highly turbulent flows.…

The wetting dynamics of liquid particles, from coated droplets to soft capsules, holds significant technological interest. Motivated by the need to simulate liquid metal droplet with an oxidize surface layer, in this work we introduce a…

Accurate estimation of surface wettability for various degrees of hydrophobicity becomes increasingly important in the molecular design of membrane. In this paper, we develop simple yet physically realistic model for estimating contact…

We present the Seismic Laboratory for Imaging and Modeling/Monitoring (SLIM) open-source software framework for computational geophysics and, more generally, inverse problems involving the wave-equation (e.g., seismic and medical…

Computing all critical points of a monomial on a very affine variety is a fundamental task in algebraic statistics, particle physics and other fields. The number of critical points is known as the maximum likelihood (ML) degree. When the…

This paper concerns the question of how AI systems encode semantic structure into the geometric structure of their representation spaces. The motivating observation of this paper is that the natural geometry of these representation spaces…

Central to the application of many multi-view geometry algorithms is the extraction of matching points between multiple viewpoints, enabling classical tasks such as camera pose estimation and 3D reconstruction. Many approaches that…

Soft slender structures are ubiquitous in natural and artificial systems and can be observed at scales that range from the nanometric to the kilometric, from polymers to space tethers. We present a practical numerical approach to simulate…

The pseudopotential model within the Lattice Boltzmann Method (LBM) framework has emerged as a prominent approach in computational fluid dynamics due to its dual strengths in physical intuitiveness and computational tractability. However,…

Decoding in large language models is often described as scoring tokens and normalizing with softmax. We give a minimal, self-contained account of this step as a constrained variational principle on the probability simplex. The discrete,…

Soft particles display highly versatile properties with respect to hard colloids, even more so at fluid-fluid interfaces. In particular, microgels, consisting of a cross-linked polymer network, are able to deform and flatten upon adsorption…