Related papers: Softmax as a Lagrangian-Legendrian Seam
Neural language models (NLMs) have recently gained a renewed interest by achieving state-of-the-art performance across many natural language processing (NLP) tasks. However, NLMs are very computationally demanding largely due to the…
The isotropic to nematic transition in a system of soft spherocylinders is studied by means of grand canonical Monte Carlo simulations. The probability distribution of the particle density is used to determine the coexistence density of the…
In text generation, a large language model (LM) makes a choice of each new word based only on the former selection of its context using the softmax function. Nevertheless, the link statistics information of concurrent words based on a…
In this paper we address smoothing-that is, optimisation-based-estimation techniques for localisation problems in the case where motion sensors are very accurate. Our mathematical analysis focuses on the difficult limit case where motion…
Variational analysis presents a unified theory encompassing in particular both smoothness and convexity. In a Euclidean space, convex sets and smooth manifolds both have straightforward local geometry. However, in the most basic hybrid case…
We introduce an accurate and efficient method for characterizing surface wetting and interfacial properties, such as the contact angle made by a liquid droplet on a solid surface, and the vapor-liquid surface tension of a fluid. The method…
Modeling membrane interactions with arbitrarily shaped colloidal particles, such as environmental micro- and nanoplastics, at the cell scale remains particularly challenging, owing to the complexity of particle geometries and the need to…
We describe a method for modeling the geometry of porous materials. The approach enables the independent selection of crucial parameters, including porosity, pore size distribution, pore shape, and connectivity. Consequently, it can…
A very popular class of models for networks posits that each node is represented by a point in a continuous latent space, and that the probability of an edge between nodes is a decreasing function of the distance between them in this latent…
Using Legendrian immersions and, in particular, Legendre curves in odd dimensional spheres and anti De Sitter spaces, we provide a method of construction of new examples of Hamiltonian-minimal Lagrangian submanifolds in complex projective…
We study a class of Legendrian surfaces in contact five-folds by encoding their wavefronts via planar combinatorial structures. We refer to these surfaces as Legendrian weaves, and to the combinatorial objects as N-graphs. First, we develop…
The Softmax bottleneck was first identified in language modeling as a theoretical limit on the expressivity of Softmax-based models. Being one of the most widely-used methods to output probability, Softmax-based models have found a wide…
In various applications, such as virtual reality and gaming, simulating the deformation of soft tissues in the human body during interactions with external objects is essential. Traditionally, Finite Element Methods (FEM) have been employed…
A maximum likelihood methodology for the parameters of models with an intractable likelihood is introduced. We produce a likelihood-free version of the stochastic approximation expectation-maximization (SAEM) algorithm to maximize the…
From understanding the sand on the beach to the foam on your beer, soft sphere simulations have been crucial to the study of the amorphous world around us. However, many of the materials we interact with on a daily basis aren't comprised of…
Is the output softmax layer, which is adopted by most language models (LMs), always the best way to compute the next word probability? Given so many attention layers in a modern transformer-based LM, are the pointer networks redundant…
We describe the recent developments to extend the multi-parton interaction model of underlying events in Herwig++ into the soft, non-perturbative, regime. This allows the program to describe also minimum bias collisions in which there is no…
Complex models in physics, biology, economics, and engineering are often sloppy, meaning that the model parameters are not well determined by the model predictions for collective behavior. Many parameter combinations can vary over decades…
Using a particle model of Physarum displaying emer- gent morphological adaptation behaviour we demonstrate how a minimal approach to collective material computation may be used to transform and summarise properties of spatially represented…
In recent years, soft robotics simulators have evolved to offer various functionalities, including the simulation of different material types (e.g., elastic, hyper-elastic) and actuation methods (e.g., pneumatic, cable-driven, servomotor).…