Related papers: A completely uniform transformer for parity
This paper demonstrates that any general 3-DOF three-legged planar parallel robot with extensible legs can change assembly modes without passing through parallel singularities (configurations where the mobile platform loses its stiffness).…
Recent work has shown that Transformers trained from scratch can successfully solve various arithmetic and algorithmic tasks, such as adding numbers and computing parity. While these Transformers generalize well on unseen inputs of the same…
We present Unicoder, a universal language encoder that is insensitive to different languages. Given an arbitrary NLP task, a model can be trained with Unicoder using training data in one language and directly applied to inputs of the same…
Transformer networks have seen great success in natural language processing and machine vision, where task objectives such as next word prediction and image classification benefit from nuanced context sensitivity across high-dimensional…
Previous work has suggested that parameter sharing between transition-based neural dependency parsers for related languages can lead to better performance, but there is no consensus on what parameters to share. We present an evaluation of…
This paper investigates the decoding of a remarkable set of lattices: We treat in a unified framework the Leech lattice in dimension 24, the Nebe lattice in dimension 72, and the Barnes-Wall lattices. A new interesting lattice is…
The recent surge in Time Series Foundation Models has rapidly advanced the field, yet the heterogeneous training setups across studies make it difficult to attribute improvements to architectural innovations versus data engineering. In this…
We show that conformal blocks simplify greatly when there is a large difference between two of the scaling dimensions for external operators. In particular the spacetime dimension only appears in an overall constant which we determine via…
The linkage construction and its generalization is one of the most powerful constructions for constant dimension code, accounting for approximately 50\% of all the listed parameters. We show how to improve the linkage construction of…
We propose a new positional encoding method for a neural network architecture called the Transformer. Unlike the standard sinusoidal positional encoding, our approach is based on solid mathematical grounds and has a guarantee of not losing…
Auto Composing is an active and appealing research area in the past few years, and lots of efforts have been put into inventing more robust models to solve this problem. With the fast evolution of deep learning techniques, some deep neural…
Understanding Transformer-based models has attracted significant attention, as they lie at the heart of recent technological advances across machine learning. While most interpretability methods rely on running models over inputs, recent…
This paper introduces a class of structured lowdensity parity-check (LDPC) codes whose parity check matrices are arrays of permutation matrices. The permutation matrices are obtained from Latin squares and form a finite field under some…
It is pointed out that if we allow for the possibility of a multilayered universe, it is possible to maintain exact supersymmetry and arrange, in principle, for the vanishing of the cosmological constant. Superpartner(s) of a known particle…
A recent variation of Transformer, Performer, scales Transformer to longer sequences with a linear attention mechanism. However, it is not compatible with relative position encoding, which has advantages over absolute position encoding. In…
Large Transformer models yield impressive results on many tasks, but are expensive to train, or even fine-tune, and so slow at decoding that their use and study becomes out of reach. We address this problem by leveraging sparsity. We study…
Cross-lingual semantic textual similarity systems estimate the degree of the meaning similarity between two sentences, each in a different language. State-of-the-art algorithms usually employ machine translation and combine vast amount of…
By considering the parity-transformation properties of the $(1/2,\,0)$ and $(0,\,1/2)$ fields in the {\it front form} we find ourselves forced to study the front-form evolution both along $x^+$ and $x^-$ directions. As a by product, we find…
In this paper, we study the absolute equivalence between Pfaffian systems with a degree 1 independence condition and obtain structural results, particularly for systems of corank 3. We apply these results to understanding dynamic feedback…
Recently, low-resolution LDPC decoders have been introduced that perform mutual information maximizing signal processing. However, the optimal quantization in variable and check nodes requires expensive non-uniform operations. Instead, we…