Related papers: De Bruijn entropy and string similarity

This paper is concerned with the approximation of the compressible Euler equations supplemented with an arbitrary or tabulated equation of state. The proposed approximation technique is robust, formally second-order accurate in space,…

In many applications, it is necessary to determine the similarity of two strings. A widely-used notion of string similarity is the edit distance: the minimum number of insertions, deletions, and substitutions required to transform one…

Categorical compositional distributional model of Coecke et al. (2010) suggests a way to combine grammatical composition of the formal, type logical models with the corpus based, empirical word representations of distributional semantics.…

The definition of $k^{th}$-order empirical entropy of strings is extended to node labelled binary trees. A suitable binary encoding of tree straight-line programs (that have been used for grammar-based tree compression before) is shown to…

We consider a symmetric mixture of linear regressions with random samples from the pairwise comparison design, which can be seen as a noisy version of a type of Euclidean distance geometry problem. We analyze the expectation-maximization…

We develop a family of techniques to align word embeddings which are derived from different source datasets or created using different mechanisms (e.g., GloVe or word2vec). Our methods are simple and have a closed form to optimally rotate,…

We consider the entanglement entropy for a spacetime region and its spacelike complement in the framework of algebraic quantum field theory. For a M\"obius covariant local net satisfying a certain nuclearity property, we consider the von…

This work examines the possibility of using syllable embeddings, instead of the often used $n$-gram embeddings, as subword embeddings. We investigate this for two languages: English and Dutch. To this end, we also translated two standard…

Generalising a recent work of Dequ\^ene et al. on the connection between perfectly clustering words and band bricks over a particular family of gentle algebras, we characterise band bricks over string algebras whose underlying quiver is…

We present an unsupervised approach for discovering semantic representations of mathematical equations. Equations are challenging to analyze because each is unique, or nearly unique. Our method, which we call equation embeddings, finds good…

The depinning of an elastic line interacting with a quenched disorder is studied for long range interactions, applicable to crack propagation or wetting. An ultrametric distance is introduced instead of the Euclidean distance, allowing for…

Ambiguity is ubiquitous in natural language. Resolving ambiguous meanings is especially important in information retrieval tasks. While word embeddings carry semantic information, they fail to handle ambiguity well. Transformer models have…

As large language models continue to scale, their growing computational and storage demands pose significant challenges for real-world deployment. In this work, we investigate redundancy within Transformer-based models and propose an…

Word embeddings are a popular way to improve downstream performances in contemporary language modeling. However, the underlying geometric structure of the embedding space is not well understood. We present a series of explorations using…

The Chapter starts with introductory information about quantitative linguistics notions, like rank--frequency dependence, Zipf's law, frequency spectra, etc. Similarities in distributions of words in texts with level occupation in quantum…

Let k be an imaginary quadratic number field (with class number 1). We describe a new, essentially linear-time algorithm, to list all isomorphism classes of cubic extensions L/k up to a bound X on the norm of the relative discriminant…

This thesis develops a new divergence that generalizes relative entropy and can be used to compare probability measures without a requirement of absolute continuity. We establish properties of the divergence, and in particular derive and…

Consanguinity of entropy and complexity is pointed out through the example of coherent states of the group $SL(d+1,\C)$. Both are obtained from the K\"ahler potential of the underlying geometry of the sphere corresponding to the…

The goal of this paper is to explore the basic Approximate Bayesian Computation (ABC) algorithm via the lens of information theory. ABC is a widely used algorithm in cases where the likelihood of the data is hard to work with or…

Edit-distance-based string similarity search has many applications such as spell correction, data de-duplication, and sequence alignment. However, computing edit distance is known to have high complexity, which makes string similarity…