Related papers: Subspace Embeddings Under Nonlinear Transformation…

Why does the low dimensionality of representations, typically $d\approx 1000$, not prevent modern embedding-based retrieval models from scaling to billions, or even trillions, of data points? To answer this question, we study maximal-margin…

Transformer models learn to encode and decode an input text, and produce contextual token embeddings as a side-effect. The mapping from language into the embedding space maps words expressing similar concepts onto points that are close in…

In this paper we study {\em terminal embeddings}, in which one is given a finite metric $(X,d_X)$ (or a graph $G=(V,E)$) and a subset $K \subseteq X$ of its points are designated as {\em terminals}. The objective is to embed the metric into…

Let $P_{\alpha} f(x,t)$ be the Caffarelli-Silvestre extension of a smooth function $f(x): \mathbb{R}^n \rightarrow \mathbb{R}^{n+1}_+:=\mathbb{R}^n\times (0,\infty).$ The purpose of this article is twofold. Firstly, we want to characterize…

Building on recent advances in representation learning for wireless channels, this work investigates the cost-benefit trade-offs of high-dimensional channel embeddings in practical systems. We benchmark multiple wireless representations:…

We consider the problem of embedding a finite set of points $\{x_1, \ldots, x_n\} \in \mathbb{R}^d$ that satisfy $\ell_2^2$ triangle inequalities into $\ell_1$, when the points are approximately low-dimensional. Goemans (unpublished,…

We develop new tools in the theory of nonlinear random matrices and apply them to study the performance of the Sum of Squares (SoS) hierarchy on average-case problems. The SoS hierarchy is a powerful optimization technique that has achieved…

Sentence embeddings can be decoded to give approximations of the original texts used to create them. We explore this effect in the context of text simplification, demonstrating that reconstructed text embeddings preserve complexity levels.…

Performance optimization is an increasingly challenging but often repetitive task. While each platform has its quirks, the underlying code transformations rely on data movement and computational characteristics that recur across…

Deep Metric Learning (DML) serves to learn an embedding function to project semantically similar data into nearby embedding space and plays a vital role in many applications, such as image retrieval and face recognition. However, the…

Representation learning (RL) methods learn objects' latent embeddings where information is preserved by distances. Since distances are invariant to certain linear transformations, one may obtain different embeddings while preserving the…

In this work, we discuss low-parametric approaches for approximating SimRank matrices, which estimate the similarity between pairs of nodes in a graph. Although SimRank matrices and their computation require a significant amount of memory,…

Let $X$ be a set of $n$ points of norm at most $1$ in the Euclidean space $R^k$, and suppose $\varepsilon>0$. An $\varepsilon$-distance sketch for $X$ is a data structure that, given any two points of $X$ enables one to recover the square…

Sobolev-type embeddings on metric measure spaces encode a subtle interaction between the analytic regularity of functions and the geometry of the underlying domain space. In this paper we develop an embedding theory for variable…

The planar embedding conjecture asserts that any planar metric admits an embedding into L_1 with constant distortion. This is a well-known open problem with important algorithmic implications, and has received a lot of attention over the…

In this note we study the problem of sampling and reconstructing signals which are assumed to lie on or close to one of several subspaces of a Hilbert space. Importantly, we here consider a very general setting in which we allow infinitely…

Different studies of the embedding space of transformer models suggest that the distribution of contextual representations is highly anisotropic - the embeddings are distributed in a narrow cone. Meanwhile, static word representations…

Data dimensionality reduction in radio interferometry can provide savings of computational resources for image reconstruction through reduced memory footprints and lighter computations per iteration, which is important for the scalability…

This paper presents a new technique for creating monolingual and cross-lingual meta-embeddings. Our method integrates multiple word embeddings created from complementary techniques, textual sources, knowledge bases and languages. Existing…

We present hash embeddings, an efficient method for representing words in a continuous vector form. A hash embedding may be seen as an interpolation between a standard word embedding and a word embedding created using a random hash function…