Related papers: Position Coding
A storage code is an assignment of symbols to the vertices of a connected graph $G(V,E)$ with the property that the value of each vertex is a function of the values of its neighbors, or more generally, of a certain neighborhood of the…
The attention mechanism is a critical component of Large Language Models (LLMs) that allows tokens in a sequence to interact with each other, but is order-invariant. Incorporating position encoding (PE) makes it possible to address by…
In transformer architectures, position encoding primarily provides a sense of sequence for input tokens. While the original transformer paper's method has shown satisfactory results in general language processing tasks, there have been new…
A spread code is a set of vector spaces of a fixed dimension over a finite field Fq with certain properties used for random network coding. It can be constructed in different ways which lead to different decoding algorithms. In this work we…
This paper presents a convenient graphical tool for encoding visual patterns (such as image patches and image atoms) as point constellations in a space spanned by perceptual features and with a clear geometrical interpretation. General…
For most algorithms dealing with sets of points in the plane, the only relevant information carried by the input is the combinatorial configuration of the points: the orientation of each triple of points in the set (clockwise,…
There is a class of entropy-coding methods which do not substitute symbols by code words (such as Huffman coding), but operate on intervals or ranges. This class includes three prominent members: conventional arithmetic coding, range…
Spread codes and orbit codes are special families of constant dimension subspace codes. These codes have been well-studied for their error correction capability and transmission rate, but the question of how to encode messages has not been…
We present a novel technique for encoding and decoding constant weight binary codes that uses a geometric interpretation of the codebook. Our technique is based on embedding the codebook in a Euclidean space of dimension equal to the weight…
Understanding a neural code requires knowledge both of the elementary symbols that transmit information and of the algorithm for translating these symbols into sensory signals or motor actions. We show that these questions can be separated:…
Coding in the projective space has received recently a lot of attention due to its application in network coding. Reduced row echelon form of the linear subspaces and Ferrers diagram can play a key role for solving coding problems in the…
Planes are familiar mathematical objects which lie at the subtle boundary between continuous geometry and discrete combinatorics. A plane is geometrical, certainly, but the ways that two planes can interact break cleanly into discrete sets:…
Color coding is an algorithmic technique used in parameterized complexity theory to detect "small" structures inside graphs. The idea is to derandomize algorithms that first randomly color a graph and then search for an easily-detectable,…
A constant-rate encoder--decoder pair is presented for a fairly large family of two-dimensional (2-D) constraints. Encoding and decoding is done in a row-by-row manner, and is sliding-block decodable. Essentially, the 2-D constraint is…
A barcode is a finite multiset of intervals on the real line, $B = \{ (b_i, d_i)\}_{i=1}^n$. Barcodes are important objects in topological data analysis, where they serve as summaries of the persistent homology groups of a filtration. The…
Linear-programming pseudocodewords play a pivotal role in our understanding of the linear-programming decoding algorithms. These pseudocodewords are known to be equivalent to the graph-cover pseudocodewords. The latter pseudocodewords, when…
Recent studies show that paddings in convolutional neural networks encode absolute position information which can negatively affect the model performance for certain tasks. However, existing metrics for quantifying the strength of…
The surface code is a two-dimensional topological code with code parameters that scale optimally with the number of physical qubits, under the constraint of two-dimensional locality. In three spatial dimensions an analogous simple yet…
Motivated by signal processing, we present a new class of channel codes, called signal codes, for continuous-alphabet channels. Signal codes are lattice codes whose encoding is done by convolving an integer information sequence with a fixed…
Transformer architectures rely on position encodings to model the spatial structure of input data. Rotary Position Encoding (RoPE) is a widely used method in language models that encodes relative positions through fixed, block-diagonal,…