Related papers: De Bruijn entropy and string similarity
We introduce the algebraic entropy for endomorphisms of arbitrary abelian groups, appropriately modifying existing notions of entropy. The basic properties of the algebraic entropy are given, as well as various examples. The main result of…
We propose a semantic similarity metric for image registration. Existing metrics like euclidean distance or normalized cross-correlation focus on aligning intensity values, giving difficulties with low intensity contrast or noise. Our…
Any physical system can be viewed from the perspective that information is implicitly represented in its state. However, the quantification of this information when it comes to complex networks has remained largely elusive. In this work, we…
We study the problem of edit similarity joins, where given a set of strings and a threshold value $K$, we want to output all pairs of strings whose edit distances are at most $K$. Edit similarity join is a fundamental problem in data…
Current work in lexical distributed representations maps each word to a point vector in low-dimensional space. Mapping instead to a density provides many interesting advantages, including better capturing uncertainty about a representation…
In this paper, we present an efficient algorithm for solving a linear optimization problem with entropic constraints, a class of problems that arises in game theory and information theory. Our analysis distinguishes between the cases of…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
We develop a maximum relative entropy formalism to generate optimal approximations to probability distributions. The central results consist in (a) justifying the use of relative entropy as the uniquely natural criterion to select a…
This paper deals with a method for the approximation of a spectral density function among the solutions of a generalized moment problem a` la Byrnes/Georgiou/Lindquist. The approximation is pursued with respect to the Kullback-Leibler…
Uncertainty estimation remains a key challenge when adapting pre-trained language models to downstream classification tasks, with overconfidence often observed for difficult inputs. While predictive entropy provides a strong baseline for…
Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over…
Let $\widetilde{\alpha}$ be a length-$L$ cyclic sequence of characters from a size-$K$ alphabet $\mathcal{A}$ such that the number of occurrences of any length-$m$ string on $\mathcal{A}$ as a substring of $\widetilde{\alpha}$ is $\lfloor L…
An experimental approach to studying the properties of word embeddings is proposed. Controlled experiments, achieved through modifications of the training corpus, permit the demonstration of direct relations between word properties and word…
A highly anticipated use of quantum computers is the simulation of complex quantum systems including molecules and other many-body systems. One promising method involves directly applying a linear combination of unitaries (LCU) to…
Recent advancements in quantum computing and quantum-inspired algorithms have sparked renewed interest in binary optimization. These hardware and software innovations promise to revolutionize solution times for complex problems. In this…
Sequence classification algorithms, such as SVM, require a definition of distance (similarity) measure between two sequences. A commonly used notion of similarity is the number of matches between $k$-mers ($k$-length subsequences) in the…
We study the tractability of conjunctive query answering for queries with unbounded arity. It is well known that tractability of the problem can be characterised in terms of the queries treewidth under the assumption of bounded arity. We…
We present a method which displays all palindromes of a given length from De Bruijn words of a certain order, and also a recursive one which constructs all palindromes of length $n+1$ from the set of palindromes of length $n$. We show that…
Entity resolution is a widely studied problem with several proposals to match records across relations. Matching textual content is a widespread task in many applications, such as question answering and search. While recent methods achieve…
To improve word representation learning, we propose a probabilistic prior which can be seamlessly integrated with word embedding models. Different from previous methods, word embedding is taken as a probabilistic generative model, and it…