Related papers: Position Coding
We introduce a unified generalization of several well-established high-throughput coding techniques including staircase codes, tiled diagonal zipper codes, continuously interleaved codes, open forward error correction (OFEC) codes, and…
Recent studies have demonstrated the effectiveness of position encoding in transformer architectures. By incorporating positional information, this approach provides essential guidance for modeling dependencies between elements across…
Vision Transformers have achieved remarkable success in computer vision, but their common use of learnable one-dimensional positional encodings weakens the inherent two-dimensional spatial structure of images after patch flattening.…
Subspace codes form the appropriate mathematical setting for investigating the Koetter-Kschischang model of fault-tolerant network coding. The Main Problem of Subspace Coding asks for the determination of a subspace code of maximum size…
In order to preserve word-order information in a non-autoregressive setting, transformer architectures tend to include positional knowledge, by (for instance) adding positional encodings to token embeddings. Several modifications have been…
This paper introduces Claycode, a novel 2D scannable code designed for extensive stylization and deformation. Unlike traditional matrix-based codes (e.g., QR codes), Claycodes encode their message in a tree structure. During the encoding…
We propose a novel learning-based formulation for visual localization of vehicles that can operate in real-time in city-scale environments. Visual localization algorithms determine the position and orientation from which an image has been…
Word order, an essential property of natural languages, is injected in Transformer-based neural language models using position encoding. However, recent experiments have shown that explicit position encoding is not always useful, since some…
The ``losing positions" of certain combinatorial games constitute linear error detecting and correcting codes. We show that a large class of games that can be cast in the form of *annihilation games*, provides a potentially polynomial…
Consider a smooth point O of a complex analytic surface S. A constellation based at O is a set of infinitely near points of O, centers of a sequence of blow-ups above O. Finite constellations are usually encoded in two ways: either using an…
Transformers, central to the successes in modern Natural Language Processing, often falter on arithmetic tasks despite their vast capabilities --which paradoxically include remarkable coding abilities. We observe that a crucial challenge is…
A textile structure is a periodic arrangement of threads in the thickened plane. A topological classification of textile structures is harder than for classical knots and links that are non-periodic and restricted to a bounded region. The…
Polynomial remainder codes are a large class of codes derived from the Chinese remainder theorem that includes Reed-Solomon codes as a special case. In this paper, we revisit these codes and study them more carefully than in previous work.…
Basic algebraic and combinatorial properties of finite vector spaces in which individual vectors are allowed to have multiplicities larger than $ 1 $ are derived. An application in coding theory is illustrated by showing that multispace…
Implicit neural representations (INRs) are increasingly being used as tools to map coordinates to signals, encompassing applications from neural fields to texture compression, shape representations, and beyond. Most INR methods are based on…
We define and study a class of codes obtained from scrolls over curves of any genus over finite fields. These codes generalize Goppa codes in a natural way, and the orthogonal complements of these codes belong to the same class. We show how…
Sparse coding, which represents a data point as a sparse reconstruction code with regard to a dictionary, has been a popular data representation method. Meanwhile, in database retrieval problems, learning the ranking scores from data points…
This paper describes how to apply self-attention with relative positional encodings to the task of relation extraction. We propose to use the self-attention encoder layer together with an additional position-aware attention layer that takes…
This paper focuses on a particular transmission scheme called local network coding, which has been reported to provide significant performance gains in practical wireless networks. The performance of this scheme strongly depends on the…
The theoretical code-switching (CS) literature provides numerous pointwise investigations that aim to explain patterns in CS, i.e. why bilinguals switch language in certain positions in a sentence more often than in others. A resulting…