Related papers: Universal Compression of Power-Law Distributions
One of the first steps to understand and forecast economic downturns is identifying their frequency distribution, but it remains uncertain. This problem is common in phenomena displaying power-law-like distributions. Power laws play a…
We propose and study a family of universal sequential probability assignments on individual sequences, based on the incremental parsing procedure of the Lempel-Ziv (LZ78) compression algorithm. We show that the normalized log loss under any…
Tokenization is a fundamental step in natural language processing (NLP) and other sequence modeling domains, where the choice of vocabulary size significantly impacts model performance. Despite its importance, selecting an optimal…
Scientific cooperation on an international level has been well studied in the literature. However, much less is known about this cooperation on the intercontinental level. In this paper, we address this issue by creating a collection of…
Chance constrained optimal power flow (CC-OPF) formulations have been proposed to minimize operational costs while controlling the risk arising from uncertainties like renewable generation and load consumption. To solve CC-OPF, we often…
We first describe a general class of optimization problems that describe many natural, economic, and statistical phenomena. After noting the existence of a conserved quantity in a transformed coordinate system, we outline several instances…
This article presents the calculation of the entropy of a system with Zipfian distribution and shows that a communication system tends to present an exponent value close to one, but still greater than one, so that it might maximize entropy…
It is shown that the distribution of low variability periods in the activity of human heart rate typically follows a multi-scaling Zipf's law. The presence or failure of a power law, as well as the values of the scaling exponents, are…
A number of writers have supposed that for the full specification of belief, higher order probabilities are required. Some have even supposed that there may be an unending sequence of higher order probabilities of probabilities of…
The power law is useful in describing count phenomena such as network degrees and word frequencies. With a single parameter, it captures the main feature that the frequencies are linear on the log-log scale. Nevertheless, there have been…
[Draft] In this paper, the redundancy of Slepian Wolf coding is revisited. Applying the random binning and converse technique in \cite{yang}, the same results in \cite{he} are obtained with much simpler proofs. Moreover, our results reflect…
Random multiplicative processes $w_t =\lambda_1 \lambda_2 ... \lambda_t$ (with < \lambda_j > 0 ) lead, in the presence of a boundary constraint, to a distribution $P(w_t)$ in the form of a power law $w_t^{-(1+\mu)}$. We provide a simple and…
Network redundancy is one of the spatial network structural properties critical to robustness against cascading failures in power networks. The waiting-time distributions for network partitions in cascading failures explain how the spatial…
This paper presents new lower and upper bounds for the optimal compression of binary prefix codes in terms of the most probable input symbol, where compression efficiency is determined by the nonlinear codeword length objective of…
Zipf's law states that the frequency of an observation with a given value is inversely proportional to the square of that value; Taylor's law, instead, describes the scaling between fluctuations in the size of a population and its mean.…
This article was written for the Logic in Computer Science column in the February 2015 issue of the Bulletin of the European Association for Theoretical Computer Science. The intended audience is general computer science audience. The…
A compression function is a map that slims down an observational set into a subset of reduced size, while preserving its informational content. In multiple applications, the condition that one new observation makes the compressed set change…
The trapping redundancy of a linear code is the number of rows of a smallest parity-check matrix such that no submatrix forms an $(a,b)$-trapping set. This concept was first introduced in the context of low-density parity-check (LDPC) codes…
Extinction times in resampling processes are fundamental yet often intractable, as previous formulas scale as $2^M$ with the number of states $M$ present in the initial probability distribution. We solve this by treating multinomial updates…
Transformers used for evidence-grounded question answering with binary adjudication (e.g., support/refute or yes/no) can be highly sensitive to the order in which exchangeable evidence is presented, producing dispersion across permutations…