Related papers: Diffraction of Random Noble Means Words

Diffusion models do not recover semantic structure uniformly over time. Instead, samples transition from semantic ambiguity to class commitment within a narrow regime. Recent theoretical work attributes this transition to dynamical…

We show how geometric methods from the general theory of fractal dimensions and iterated function systems can be deployed to study symbolic dynamics in the zero entropy regime. More precisely, we establish a dimensional characterization of…

We consider a situation where the distribution of a random variable is being estimated by the empirical distribution of noisy measurements of that variable. This is common practice in, for example, teacher value-added models and other…

We find a general formula for the distribution of time averaged observables for weakly non-ergodic systems. Such type of ergodicity breaking is known to describe certain systems which exhibit anomalous fluctuations, e.g. blinking quantum…

The time variation of entropy, as an alternative to the variance, is proposed as a measure of the diffusion rate. It is shown that for linear and time-translationally invariant systems having a large-time limit for the density, at large…

We study the spectrum of a random matrix, whose elements depend on the Euclidean distance between points randomly distributed in space. This problem is widely studied in the context of the Instantaneous Normal Modes of fluids and is…

How does word frequency in pre-training data affect the behavior of similarity metrics in contextualized BERT embeddings? Are there systematic ways in which some word relationships are exaggerated or understated? In this work, we explore…

To address the challenge of quantifying uncertainty in the outputs generated by language models, we propose a novel measure of semantic uncertainty, semantic spectral entropy, that is statistically consistent under mild assumptions. This…

Given a sequence composed of a limit number of characters, we try to "read" it as a "text". This involves to segment the sequence into "words". The difficulty is to distinguish good segmentation from enormous number of random ones.Aiming at…

Thanks to the increasing availability in computing power, high-dimensional engineering problems seem to be at reach. But the curse of dimensionality will always prevent us to try out extensively all the hypotheses. There is a vast…

In essence, embedding algorithms work by optimizing the distance between a word and its usual context in order to generate an embedding space that encodes the distributional representation of words. In addition to single words or word…

We introduce a class of discrete random walk model driven by global memory effects. At any time the right-left transitions depend on the whole previous history of the walker, being defined by an urn-like memory mechanism. The characteristic…

The problem of estimating the eddy diffusivity from Lagrangian observations in the presence of measurement error is studied in this paper. We consider a class of incompressible velocity fields for which is can be rigorously proved that the…

Porosity and dimension are two useful, but different, concepts that quantify the size of fractal sets and measures. An active area of research concerns understanding the relationship between these two concepts. In this article we will…

Anomaly localization in images -- identifying regions that deviate from normal patterns -- is vital in applications such as medical diagnosis and industrial inspection. A recent trend is the use of image generation models in anomaly…

Existing training criteria in automatic speech recognition(ASR) permit the model to freely explore more than one time alignments between the feature and label sequences. In this paper, we use entropy to measure a model's uncertainty, i.e.…

In this note we discuss a common misconception, namely that embeddings are always used to reduce the dimensionality of the item space. We show that when we measure dimensionality in terms of information entropy then the embedding of sparse…

In this article, we discuss subgeometric ergodicity of a class of regime-switching diffusion processes. We derive conditions on the drift and diffusion coefficients, and the switching mechanism which result in subgeometric ergodicity of the…

The diffraction spectrum of coherent waves scattered from fractal supports is calculated exactly. The fractals considered are of the class generated iteratively by successive dilations and translations, and include generalizations of the…

Many complex generative systems use languages to create structured objects. We consider a model of random languages, defined by weighted context-free grammars. As the distribution of grammar weights broadens, a transition is found from a…