Related papers: Position Coding
The location of quantum information in various subsets of the qudit carriers of an additive graph code is discussed using a collection of operators on the coding space which form what we call the information group. It represents the input…
We introduce STRING: Separable Translationally Invariant Position Encodings. STRING extends Rotary Position Encodings, a recently proposed and widely used algorithm in large language models, via a unifying theoretical framework.…
The aim of this paper is to investigate an attempt to build a binary classification algorithm using principles of geometry such as vectors, planes, and vector algebra. The basic idea behind the proposed algorithm is that a hyperplane can be…
Rotary Position Embedding (RoPE) is the de facto positional encoding in large language models due to its ability to encode relative positions and support length extrapolation. When adapted to vision transformers, the standard axial…
An index code is said to be locally decodable if each receiver can decode its demand using its side information and by querying only a subset of the transmitted codeword symbols instead of observing the entire codeword. Local decodability…
A classical method of constructing a linear code over $\gf(q)$ with a $t$-design is to use the incidence matrix of the $t$-design as a generator matrix over $\gf(q)$ of the code. This approach has been extensively investigated in the…
Bar charts are among the most frequently used visualizations, in part because their position encoding leads them to convey data values precisely. Yet reproductions of single bars or groups of bars within a graph can be biased. Curiously,…
In this paper, we generalize the well-known index coding problem to exploit the structure in the source-data to improve system throughput. In many applications, the data to be transmitted may lie (or can be well approximated) in a…
In prefix coding over an infinite alphabet, methods that consider specific distributions generally consider those that decline more quickly than a power law (e.g., Golomb coding). Particular power-law distributions, however, model many…
In this work, we study two types of constraints on two-dimensional binary arrays. In particular, given $p,\epsilon>0$, we study (i) The $p$-bounded constraint: a binary vector of size $m$ is said to be $p$-bounded if its weight is at most…
Satellite networks provide unique challenges that can restrict users' quality of service. For example, high packet erasure rates and large latencies can cause significant disruptions to applications such as video streaming or voice-over-IP.…
In this paper we present the results of computer searches using a variation of an energy minimization algorithm used by Kottwitz for finding good spherical codes. We prove that exact codes exist by representing the inner products between…
Positional Encodings (PEs) are a critical component of Transformer-based Large Language Models (LLMs), providing the attention mechanism with important sequence-position information. One of the most popular types of encoding used today in…
Transformer-based Large Language Models (LLMs) rely on positional encodings to provide sequence position information to their attention mechanism. Rotary Positional Encodings (RoPE), which encode relative position by rotating queries and…
The coding space of protein sequences is shaped by evolutionary constraints set by requirements of function and stability. We show that the coding space of a given protein family--the total number of sequences in that family--can be…
A permutation-invariant code on m qubits is a subspace of the symmetric subspace of the m qubits. We derive permutation-invariant codes that can encode an increasing amount of quantum information while suppressing leading order spontaneous…
We present results on coding using multisets instead of ordered sequences. The study is motivated by a moving object tracking problem in a sensor network and can find applications in settings where the order of the symbols in a codeword…
Learning to encode differences in the geometry and (topological) structure of the shapes of ordinary objects is key to generating semantically plausible variations of a given shape, transferring edits from one shape to another, and many…
We define a Johnson graph code as a subspace of labelings of the vertices in a Johnson graph with the property that labelings are uniquely determined by their restriction to vertex neighborhoods specified by the parameters of the code. We…
In this work we explore possibilities for coding when information worlds have different (semantic) values. We introduce a loss function that expresses the overall performance of a coding scheme for discrete channels and exchange the usual…