相关论文: Swap distance minimization beyond entropy minimiza…

A pattern $\alpha$ is a string of variables and terminal letters. We say that $\alpha$ matches a word $w$, consisting only of terminal letters, if $w$ can be obtained by replacing the variables of $\alpha$ by terminal words. The matching…

What is the smallest number of random transpositions (meaning that we swap given pairs of elements with given probabilities) that we can make on an $n$-point set to ensure that each element is uniformly distributed -- in the sense that the…

We apply two variations of the principle of Minimum Cross Entropy (the Kullback information measure) to fit parameterized probability density models to observed data densities. For an array beamforming problem with P incident narrowband…

We introduce and analyze a discrete-time network process in which each node holds a (weak) preference ordering over a finite set of alternatives and updates by local Borda aggregation. At each step, a node forms a weighted average…

To better understand the process by which humans make navigation decisions when tasked with multiple stopovers, we analyze motion data captured from shoppers in a grocery store. We discover several trends in the data that are consistent…

In two previous experiments we investigated the neural precursors of subjects' "free" choices for one of two options (pressing one of two buttons, and choosing between adding and subtracting numbers). In these experiments the distribution…

Despite the recent popularity of word embedding methods, there is only a small body of work exploring the limitations of these representations. In this paper, we consider one aspect of embedding spaces, namely their stability. We show that…

Previous work examining the Uniform Information Density (UID) hypothesis has shown that while information as measured by surprisal metrics is distributed more or less evenly across documents overall, local discrepancies can arise due to…

Individuals of different types, may it be genetic, cultural, or else, with different levels of fitness often compete for reproduction and survival. A fitter type generally has higher chances of disseminating their copies to other…

We propose an algorithm for the Wireless Sensor Network localization problem, which is based on the well-known algorithmic framework of Alternating Minimization. We start with a non-smooth and non-convex minimization, and transform it into…

Conditional mutual information is important in the selection and interpretation of graphical models. Its empirical version is well known as a generalised likelihood ratio test and that it may be represented as a difference in entropy. We…

Measure-theoretic slow entropy is a more refined invariant than the classical measure-theoretic entropy to characterize the complexity of dynamical systems with subexponential growth rates of distinguishable orbit types. In this paper we…

Long-range correlations are found in symbolic sequences from human language, music and DNA. Determining the span of correlations in dolphin whistle sequences is crucial for shedding light on their communicative complexity. Dolphin whistles…

With Zipf's law being originally and most famously observed for word frequency, it is surprisingly limited in its applicability to human language, holding over no more than three to four orders of magnitude before hitting a clear break in…

Comparison-based algorithms are algorithms for which the execution of each operation is solely based on the outcome of a series of comparisons between elements. Comparison-based computations can be naturally represented via the following…

Consider a state of a system with several subsystems. The entropies of the reduced state on different subsystems obey certain inequalities, provided there is an equivalence relation, and a function measuring volumes or weights of…

We define a notion of rank for words and subshifts that we call spacer rank, extending the notion of rank-one symbolic shifts of Gao and Hill. We construct infinite words of each finite spacer rank, of unbounded spacer rank, and show there…

Generative neural conversational systems are generally trained with the objective of minimizing the entropy loss between the training "hard" targets and the predicted logits. Often, performance gains and improved generalization can be…

The recent dramatic increase in online data availability has allowed researchers to explore human culture with unprecedented detail, such as the growth and diversification of language. In particular, it provides statistical tools to explore…

A family of information theoretic models of communication was introduced more than a decade ago to explain the origins of Zipf's law for word frequencies. The family is a based on a combination of two information theoretic principles:…