Related papers: The Rank-Size Scaling Law and Entropy-Maximizing P…
We analyse the statistical properties of public transport networks. These networks are defined by a set of public transport routes (bus lines) and the stations serviced by these. For larger networks these appear to possess a scale-free…
We checked that the distribution of words in text should uniform, which gives Heaps' law as natural result, that is, the number of types of words can be expressed as a power law of the number of tokens within text. We developed a…
Neural scaling laws aim to characterize how out-of-sample error behaves as a function of model and training dataset size. Such scaling laws guide allocation of a computational resources between model and data processing to minimize error.…
In this paper, we study the maximum entropy sampling problem (MESP) and its variants. MESP seeks to identify a small subset of variables that maximizes the determinant of a covariance submatrix, and is a fundamental model in optimal…
For the first time ever, we analyze a unique public procurement database, which includes information about a number of bidders for a contract, a final price, an identification of a winner and an identification of a contracting authority for…
Understanding how size influences the internal characteristics of a system is a crucial concern across various fields. Concepts like scale invariance, universalities, and fractals are fundamental to this inquiry and find application in…
The different between the inverse power function and the negative exponential function is significant. The former suggests a complex distribution, while the latter indicates a simple distribution. However, the association of the power-law…
This article investigates scaling laws within language families using data from over six thousand languages and analyzing emergent patterns observed in Zipf-like classification graphs. Both macroscopic (based on number of languages by…
Understanding the innovation process, that is the underlying mechanisms through which novelties emerge, diffuse and trigger further novelties is undoubtedly of fundamental importance in many areas (biology, linguistics, social science and…
We study analytically the corrections to the leading terms in the Renyi entropy of a massive lattice theory, showing significant deviations from naive expectations. In particular, we show that finite size and finite mass effects give rise…
Scaling laws play an instrumental role in the sustainable improvement in model quality. Unfortunately, recommendation models to date do not exhibit such laws similar to those observed in the domain of large language models, due to the…
When the probability of measuring a particular value of some quantity varies inversely as a power of that value, the quantity is said to follow a power law, also known variously as Zipf's law or the Pareto distribution. Power laws appear…
The cross-entropy scaling law has long served as a key tool for guiding the development of large language models. It shows that cross-entropy loss decreases in a predictable power-law rate as the model size increases. However, recent…
On a variety of tasks, the performance of neural networks predictably improves with training time, dataset size and model size across many orders of magnitude. This phenomenon is known as a neural scaling law. Of fundamental importance is…
English words and the outputs of many other natural processes are well-known to follow a Zipf distribution. Yet this thoroughly-established property has never been shown to help compress or predict these important processes. We show that…
We present a broad, phenomenological picture of the distribution of the length of open space linear segments, $l$, derived from maps of 36 cities in 14 different countries. By scaling the Zipf plot of $l$, we obtain two master curves for a…
We study a few dynamical systems composed of many components whose sizes evolve according to multiplicative stochastic rules. We compare them with respect to the emergence of power laws in the size distribution of their components. We show…
In most data-scientific approaches, the principle of Maximum Entropy (MaxEnt) is used to a posteriori justify some parametric model which has been already chosen based on experience, prior knowledge or computational simplicity. In a…
Present human languages display slightly asymmetric log-normal (Gauss) distribution for size [1-3], whereas present cities follow power law (Pareto-Zipf law)[4]. Our model considers the competition between languages and that between cities…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…