Related papers: The Transpension Type: Technical Report
Contrastive visual pretraining based on the instance discrimination pretext task has made significant progress. Notably, recent work on unsupervised pretraining has shown to surpass the supervised counterpart for finetuning downstream…
Most recent research about automatic music transcription (AMT) uses convolutional neural networks and recurrent neural networks to model the mapping from music signals to symbolic notation. Based on a high-resolution piano transcription…
Upper moduli part of adjunction is introduced and its basic property are discussed. The moduli part satisfies the BP in the case of rational multiplicities and is nef in the maximal case.
We provide several crucial technical extensions of the theory of stable independence notions in accessible categories. In particular, we describe circumstances under which a stable independence notion can be transferred from a subcategory…
Modality translation is inherently under-constrained, as multiple cross-modal mappings may yield the same marginals. Recent work has shown that diffusion bridges are effective for this task. However, most existing approaches rely on fully…
Transformers have made significant strides across various artificial intelligence domains, including natural language processing, computer vision, and audio processing. This success has naturally garnered considerable interest from both…
Transformers are arguably the main workhorse in recent Natural Language Processing research. By definition a Transformer is invariant with respect to reordering of the input. However, language is inherently sequential and word order is…
Session types capture precise protocol structure in concurrent programming, but do not specify properties of the exchanged values beyond their basic type. Refinement types are a form of dependent types that can address this limitation,…
We describe a method for predicting a classification of an object given classifications of the objects in the training set, assuming that the pairs object/classification are generated by an i.i.d. process from a continuous probability…
This paper proposes a transformer over transformer framework, called Transformer$^2$, to perform neural text segmentation. It consists of two components: bottom-level sentence encoders using pre-trained transformers, and an upper-level…
This paper is devoted to study thermodynamic formalism for suspension flows defined over countable alphabets. We are mostly interested in the regularity properties of the pressure function. We establish conditions for the pressure function…
This is the final version of a series of papers uploaded in May 25, 2005. We have splitted the long last paper of the previous version in two parts to make it easier to understand. The results are essentially the same, although the…
The increasing number of protein sequences decoded from genomes is opening up new avenues of research on linking protein sequence to function with transformer neural networks. Recent research has shown that the number of known protein…
This paper proposes the use of dependent types for pragmatic phenomena such as pronoun binding and presupposition resolution as a type-theoretic alternative to formalisms such as Discourse Representation Theory and Dynamic Semantics.
Large Language Models (LLMs) have achieved impressive performance across diverse tasks but continue to struggle with learning transitive relations, a cornerstone for complex planning. To address this issue, we investigate the Multi-Token…
Species sampling processes have long served as the fundamental framework for modeling random discrete distributions and exchangeable sequences. However, data arising from distinct but related sources require a broader notion of…
In the past few years we have seen the meteoric appearance of dozens of foundation models of the Transformer family, all of which have memorable and sometimes funny, but not self-explanatory, names. The goal of this paper is to offer a…
Sufficient ultraspherical multiplier criteria are refined in such a way that they are comparable with necessary multiplier conditions. Also new necessary conditions for Jacobi multipliers are deduced which, in particular, imply known Cohen…
We study two-dimensional subshifts whose horizontal trace (a.k.a. projective subdynamics) contains only points of finite support. Our main result is a classification result for such subshifts satisfying a minimality property. As…
Refinement types are a well-studied manner of performing in-depth analysis on functional programs. The dependency pair method is a very powerful method used to prove termination of rewrite systems; however its extension to higher order…