相关论文: A geometric relation of the error introduced by sa…

Psychological research consistently finds that human ratings of words across diverse semantic scales can be reduced to a low-dimensional form with relatively little information loss. We find that the semantic associations encoded in the…

A finite geometric model of space-time (which we call the bulk) is shown to emerge as a set of error correcting codes. The bulk is encoding a set of messages located in a blow up of the Gibbons-Hoffman-Wootters (GHW) discrete phase space…

Despite encouraging recent progresses in ensemble approaches, classification methods seem to have reached a plateau in development. Further advances depend on a better understanding of geometrical and topological characteristics of point…

Understanding the behavior of black-box large language models and determining effective means of comparing their performance is a key task in modern machine learning. We consider how large language models respond to a specific query by…

The scattering transform is a multilayered, wavelet-based transform initially introduced as a model of convolutional neural networks (CNNs) that has played a foundational role in our understanding of these networks' stability and invariance…

The communications and interrelations between different locations on the Earth's surface have far-reaching implications for both social and natural systems. Effective spatial analytics ideally require a spatial representation, where…

Representation learning plays a central role in structuring internal embeddings to capture the statistical properties of language, influencing the coherence and contextual consistency of generated text. Statistical Coherence Alignment is…

Deep Generative Models are frequently used to learn continuous representations of complex data distributions using a finite number of samples. For any generative model, including pre-trained foundation models with Diffusion or Transformer…

We study the geometric structure of layer updates in deep language models. Rather than analyzing what information is encoded in intermediate representations, we ask how representations change from one layer to the next. We show that…

Geometric representations provide a principled framework for structuring the description of latent constructs and clarifying sources of uncertainty in their dimensional characterisation. We introduce a novel geometric representation of…

We study the cosine similarity of sentence transformer embeddings and observe that they are well modeled by gamma mixtures. From a fixed corpus, we measure similarities between all document embeddings and a reference query embedding.…

Large Language Models achieve next-token prediction by transporting a vectorized piece of text (prompt) across an accompanying embedding space under the action of successive transformer layers. The resulting high-dimensional trajectories…

We propose a novel geometric approach for learning bilingual mappings given monolingual embeddings and a bilingual dictionary. Our approach decouples learning the transformation from the source language to the target language into (a)…

Deep generative models like variational autoencoders approximate the intrinsic geometry of high dimensional data manifolds by learning low-dimensional latent-space variables and an embedding function. The geometric properties of these…

The quantum geometry arising in Loop Quantum Gravity has been known to semi-classically lead to generalizations of length-geometries. There have been several attempts to interpret these so called twisted geometries and understand their role…

Representing words by vectors, or embeddings, enables computational reasoning and is foundational to automating natural language tasks. For example, if word embeddings of similar words contain similar values, word similarity can be readily…

A model in statistical mechanics, characterised by the corresponding Gibbs measure, is a subset of the totality of probability distributions on the phase space. The shape of this subset, i.e., the geometry, then plays an important role in…

We demonstrate the utility of a new methodological tool, neural-network word embedding models, for large-scale text analysis, revealing how these models produce richer insights into cultural associations and categories than possible with…

How large language models (LLMs) align with the neural representation and computation of human language is a central question in cognitive science. Using representational geometry as a mechanistic lens, we addressed this by tracking…

We survey a host of results from discrete geometry that have bearing on the analysis of geometric models of approval voting. Such models view the political spectrum as a geometric space, with geometric constraints on voter preferences.…