Related papers: De Bruijn entropy and string similarity
We introduce the notion of relative volume entropy for two spacetimes with preferred compact spacelike foliations. This is accomplished by applying the notion of Kullback-Leibler divergence to the volume elements induced on spacelike…
Do word embeddings converge to learn similar things over different initializations? How repeatable are experiments with word embeddings? Are all word embedding techniques equally reliable? In this paper we propose evaluating methods for…
In this paper, we derive a new generalisation of the strong subadditivity of the entropy to the setting of general conditional expectations onto arbitrary finite-dimensional von Neumann algebras. The latter inequality, which we call…
The similarity of local atomic environments is an important concept in many machine-learning techniques which find applications in computational chemistry and material science. Here, we present and discuss a connection between the…
Information theoretic active learning has been widely studied for probabilistic models. For simple regression an optimal myopic policy is easily tractable. However, for other tasks and with more complex models, such as classification with…
We consider in this paper the problem of computing the entropy of a braid. We recall its definition and construct, for each braid, a sequence of real numbers, whose limit is its entropy. We state one conjecture about the convergence speed,…
Satisfiability of word equations is an important problem in the intersection of formal languages and algebra: Given two sequences consisting of letters and variables we are to decide whether there is a substitution for the variables that…
We study the problem of computing similarity joins under edit distance on a set of strings. Edit similarity joins is a fundamental problem in databases, data mining and bioinformatics. It finds important applications in data cleaning and…
Determining the representativeness of a point within a data cloud has recently become a desirable task in multivariate analysis. The concept of statistical depth function, which reflects centrality of an arbitrary point, appears to be…
Using the bitstring probabilities of ground states of bipartitioned ladders of Rydberg atoms, we calculate the mutual information, which is a lower bound on the corresponding bipartite von Neumann quantum entanglement entropy $S^{vN}_A$. We…
After reviewing unnormalized and normalized information distances based on incomputable notions of Kolmogorov complexity, we discuss how Kolmogorov complexity can be approximated by data compression algorithms. We argue that optimal…
Various properties of relative entropy have led to its widespread use in information theory. These properties suggest that relative entropy has a role to play in systems that attempt to perform inference in terms of probability…
Euclidean embeddings of data are fundamentally limited in their ability to capture latent semantic structures, which need not conform to Euclidean spatial assumptions. Here we consider an alternative, which embeds data as discrete…
The preservation of ambient isotopic equivalence under piecewise linear (PL) approximation for smooth knots are prominent in molecular modeling and simulation. Sufficient conditions are given regarding: (1) Hausdorff distance, and (2) a sum…
We describe a general parameterized scheme of program and constraint analyses allowing us to specify both the program specialization method known as Turchin's supercompilation and Hmelevskii's algorithm solving the quadratic word equations.…
We provide another approach to Friedland's result that the topological entropy $h$ of a symmetric nearest-neighbor subshift is computable. Instead of the previous algebraic technique, our approach is mostly combinatorial and involves only…
This paper aims to provide an unsupervised modelling approach that allows for a more flexible representation of text embeddings. It jointly encodes the words and the paragraphs as individual matrices of arbitrary column dimension with unit…
Word embedding, which encodes words into vectors, is an important starting point in natural language processing and commonly used in many text-based machine learning tasks. However, in most current word embedding approaches, the similarity…
We review the properties of the quantum relative entropy function and discuss its application to problems of classical and quantum information transfer and to quantum data compression. We then outline further uses of relative entropy to…
In many applications, it is necessary to determine the string similarity. Edit distance[WF74] approach is a classic method to determine Field Similarity. A well known dynamic programming algorithm [GUS97] is used to calculate edit distance…