相关论文: Swap distance minimization beyond entropy minimiza…

We pose the approximation problem for scalar nonnegative input-output systems via impulse response convolutions of finite order, i.e. finite order moving averages, based on repeated observations of input/output signal pairs. The problem is…

We extend the notion of estimation entropy of autonomous dynamical systems proposed by Liberzon and Mitra [1] to nonlinear dynamical systems with uncertain inputs with bounded variation. We call this new notion the {$\epsilon$}-estimation…

Many systems in nature, from ferromagnets to flocks of birds, exhibit ordering phenomena on the large scale. In physical systems order is statistically robust for large enough dimensions, with relative fluctuations due to noise vanishing…

Motivated by Bourque and Pevzner's simulation study of the parsimony method for studying genome rearrangement, Berestycki and Durrett used techniques from random graph theory to prove that the minimum parsimony distance after iterating the…

Zipf's law of abbreviation, namely the tendency of more frequent words to be shorter, has been viewed as a manifestation of compression, i.e. the minimization of the length of forms -- a universal principle of natural communication.…

Sorted data is usually easier to compress than unsorted permutations of the same data. This motivates a simple compression scheme: specify the sorted permutation of the data along with a representation of the sorted data compressed…

Many studies have shown that human languages tend to optimize for lower complexity and increased communication efficiency. Syntactic dependency distance, which measures the linear distance between dependent words, is often considered a key…

This article presents the calculation of the entropy of a system with Zipfian distribution and shows that a communication system tends to present an exponent value close to one, but still greater than one, so that it might maximize entropy…

A key aim in biology and psychology is to identify fundamental principles underpinning the behavior of animals, including humans. Analyses of human language and the behavior of a range of non-human animal species have provided evidence for…

Biological systems often consist of a small number of constituents and are therefore inherently noisy. To function effectively, these systems must employ mechanisms to constrain the accumulation of noise. Such mechanisms have been…

Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…

Cross-lingual word embeddings aim to capture common linguistic regularities of different languages, which benefit various downstream tasks ranging from machine translation to transfer learning. Recently, it has been shown that these…

Gathering the most information by picking the least amount of data is a common task in experimental design or when exploring an unknown environment in reinforcement learning and robotics. A widely used measure for quantifying the…

While natural languages differ widely in both canonical word order and word order flexibility, their word orders still follow shared cross-linguistic statistical patterns, often attributed to functional pressures. In the effort to identify…

The Random Permutation Set (RPS) is a new type of set proposed recently, which can be regarded as the generalization of evidence theory. To measure the uncertainty of RPS, the entropy of RPS and its corresponding maximum entropy have been…

During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…

Statistical regularities in human language have fascinated researchers for decades, suggesting deep underlying principles governing its evolution and information structuring for efficient communication. While Zipf's Law describes the…

We study the minimal distance between two orbit segments of length n, in a random dynamical system with sufficiently good mixing properties. This problem has already been solved in non-random dynamical system, and on average in random…

We present theoretical guarantees for an alternating minimization algorithm for the dictionary learning/sparse coding problem. The dictionary learning problem is to factorize vector samples $y^{1},y^{2},\ldots, y^{n}$ into an appropriate…

Several cycle lexicographical orders are found to describe the relative likelihood of elements of the random walks on the symmetric group generated by the conjugacy classes of transpositions, 3-cycles, and n-cycles. Spectral analysis finds…