相关论文: Swap distance minimization beyond entropy minimiza…
Existing works have studied the impacts of the order of words within natural text. They usually analyze it by destroying the original order of words to create a scrambled sequence, and then comparing the models' performance between the…
We evaluated the impact of changing the observation scale over the entropy measures for text descriptions. MIDI coded Music, computer code and two human natural languages were studied at the scale of characters, words, and at the…
Zipf's law of abbreviation, the tendency of more frequent words to be shorter, is one of the most solid candidates for a linguistic universal, in the sense that it has the potential for being exceptionless or with a number of exceptions…
Recent research has shown that static word embeddings can encode word frequency information. However, little has been studied about this phenomenon and its effects on downstream tasks. In the present work, we systematically study the…
It is often stated that human languages, as other biological systems, are shaped by cost-cutting pressures but, to what extent? Attempts to quantify the degree of optimality of languages by means of an optimality score have been scarce and…
A sorting network is a shortest path from 12...n to n...21 in the Cayley graph of S_n generated by nearest-neighbor swaps. For m<=n, consider the random m-particle sorting network obtained by choosing an n-particle sorting network uniformly…
The entropy accumulation theorem states that the smooth min-entropy of an $n$-partite system $A = (A_1, \ldots, A_n)$ is lower-bounded by the sum of the von Neumann entropies of suitably chosen conditional states up to corrections that are…
Preserving biodiversity and ecosystem stability is a challenge that can be pursued through modern statistical mechanics modeling. Here we introduce a variational maximum entropy-based algorithm to evaluate the entropy in a minimal ecosystem…
The use of null hypotheses (in a statistical sense) is common in hard sciences but not in theoretical linguistics. Here the null hypothesis that the low frequency of syntactic dependency crossings is expected by an arbitrary ordering of…
The word embedding space in neural models is skewed, and correcting this can improve task performance. We point out that most approaches for modeling, correcting, and measuring the symmetry of an embedding space implicitly assume that the…
Maximum entropy models are increasingly being used to describe the collective activity of neural populations with measured mean neural activities and pairwise correlations, but the full space of probability distributions consistent with…
Let $\sigma$ be a primitive substitution on an alphabet $\mathcal{A}$, and let $\mathcal{W}_n$ be the set of words of length $n$ determined by $\sigma$ (i.e., $w \in \mathcal{W}_n$ if $w$ is a subword of $\sigma^k(a)$ for some $a \in…
Navigating toward a known target in a noisy environment is a fundamental problem shared across biological, physical, and engineered systems. Although optimal strategies are often framed in terms of continuous, fine-grained feedback, we show…
The shortest distance between the first $n$ iterates of a typical point can be quantified with a log rule for some dynamical systems admitting Gibbs measures. We show this in two settings. For topologically mixing Markov shifts with at most…
We characterize the minimum-length sequences of independent lazy simple transpositions whose composition is a uniformly random permutation. For every reduced word of the reverse permutation there is exactly one valid way to assign…
Inspired by the swarming or flocking of animal systems we study groups of agents moving in unbounded 2D space. Individual trajectories derive from a ``bottom-up'' principle: individuals reorient to maximise their future path entropy over…
In this paper we present a random shuffling scheme to apply with adaptive sorting algorithms. Adaptive sorting algorithms utilize the presortedness present in a given sequence. We have probabilistically increased the amount of presortedness…
The goal of ordinal embedding is to represent items as points in a low-dimensional Euclidean space given a set of constraints in the form of distance comparisons like "item $i$ is closer to item $j$ than item $k$". Ordinal constraints like…
Languages employ different strategies to transmit structural and grammatical information. While, for example, grammatical dependency relationships in sentences are mainly conveyed by the ordering of the words for languages like Mandarin…
We consider the number of occurrences of subwords (non-consecutive sub-sequences) in a given word. We first define the notion of subword entropy of a given word that measures the maximal number of occurrences among all possible subwords. We…