Related papers: The Transpension Type: Technical Report
This paper synergizes the roles of adjoint in various disciplines of mathematics, sciences, and engineering. Though the materials developed and presented are not new -- as each or some could be found in (or inferred from) publications in…
We give some results about a bijection associating each permutation with a subexcedant function. This function is related to a particular decomposition of the permutation as a product of transpositions and therefore it has been called…
We construct a model of type theory enjoying parametricity from an arbitrary one. A type in the new model is a semi-cubical type in the old one, illustrating the correspondence between parametricity and cubes. Our construction works not…
As protein informatics advances rapidly, the demand for enhanced predictive accuracy, structural analysis, and functional understanding has intensified. Transformer models, as powerful deep learning architectures, have demonstrated…
The renormalization group flow of the multiscalar interacting $\varphi^3$ theory in $d=6$ dimensions is known to have a gradient structure, in which suitable generalizations of the beta functions $B^{I}$ emerge as the gradient of a scalar…
Deep learning models such as the Transformer are often constructed by heuristics and experience. To provide a complementary foundation, in this work we study the following problem: Is it possible to find an energy function underlying the…
In this work we introduce a self-supervised pretraining framework for transformers on functional Magnetic Resonance Imaging (fMRI) data. First, we pretrain our architecture on two self-supervised tasks simultaneously to teach the model a…
A property $(D)$ of subshifts was defined in: Wolfgang Krieger, On $g$-functions for subshifts, IMS Lecture Notes- Monograph Series, Vol. 48, Dynamics & Stochastics (2006) 306 - 316, arXiv:math.DS/0608259. With a view towards a theory of…
Sequence modelling requires determining which past tokens are causally relevant from the context and their importance: a process inherent to the attention layers in transformers, yet whose underlying learned mechanisms remain poorly…
The aim of these notes is to introduce the intuition motivating the notion of a "complicial set", a simplicial set with certain marked "thin" simplices that witness a composition relation between the simplices on their boundary. By varying…
This article handles in a short manner a few Laplace transform pairs and some extensions to the basic equations are developed. They can be applied to a wide variety of functions in order to find the Laplace transform or its inverse when…
Parametricity is a key metatheoretic property of type systems, which implies strong uniformity & modularity properties of the structure of types within systems possessing it. In recent years, various systems of dependent type theory have…
Predicting protein secondary structures such as alpha helices, beta sheets, and coils from amino acid sequences is essential for understanding protein function. This work presents a transformer-based model that applies attention mechanisms…
Transformers have achieved great success across a wide range of applications, yet the theoretical foundations underlying their success remain largely unexplored. To demystify the strong capacities of transformers applied to versatile…
This technical report investigates Kripke-style modal type theories, both simply typed and dependently typed. We examine basic meta-theories of the type theories, develop their substitution calculi, and give normalization by evaluation…
The Transformer is a sequence model that forgoes traditional recurrent architectures in favor of a fully attention-based approach. Besides improving performance, an advantage of using attention is that it can also help to interpret a model…
Transformers are powerful text representation learners, useful for all kinds of clinical decision support tasks. Although they outperform baselines on readmission prediction, they are not infallible. Here, we look into one such failure…
A full subcategory of a Grothendieck category is called deconstructible if it consists of all transfinite extensions of some set of objects. This concept provides a handy framework for structure theory and construction of approximations for…
We discuss the back and forth technique in the context of presheaf model theory. The essence of the back and forth technique lies in showing the relationship between various hierarchies which calibrate similarity between two models and,…
Training multimodal foundation models is challenging due to the limited availability of multimodal datasets. While many public datasets pair images with text, few combine images with audio or text with audio. Even rarer are datasets that…