Related papers: The Harmonic Structure of Information Contours
The Uniform Information Density (UID) principle posits that humans prefer to spread information evenly during language production. We examine if this UID principle can help capture differences between Large Language Models (LLMs)-generated…
With improved measurement and modelling technology, variability has emerged as an essential feature in non-equilibrium processes. While traditionally, mean values and variance have been heavily used, they are not appropriate in describing…
Language change is a cultural evolutionary process in which variants of linguistic variables change in frequency through processes analogous to mutation, selection and genetic drift. In this work, we apply a recently-introduced method to…
We propose an effect called information constraint which is characterized by the existence of different decay rates of signal strengths propagating along opposite directions. It is an intrinsic property of a type of open quantum system,…
Music comprises of a set of complex simultaneous events organized in time. In this paper we introduce a novel framework that we call Deep Musical Information Dynamics, which combines two parallel streams - a low rate latent representation…
Flexible information routing fundamentally underlies the function of many biological and artificial networks. Yet, how such systems may specifically communicate and dynamically route information is not well understood. Here we identify a…
Text-to-image diffusion models have made significant progress in generating naturalistic images from textual inputs, and demonstrate the capacity to learn and represent complex visual-semantic relationships. While these diffusion models…
The effects of demographic stochasticity in the long term behaviour of endemic infectious diseases have been considered for long as a necessary addition to an underlying deterministic theory. The latter would explain the regular behaviour…
Information scrambling, the process by which quantum information spreads and becomes effectively inaccessible, is central to modern quantum statistical physics and quantum chaos. These lecture notes provide an introduction to information…
Human perceptual and cognitive abilities are limited resources. Today, in the age of cheap information --cheap to produce, to manipulate, to disseminate--, this cognitive bottleneck translates into hypercompetition for visibility among…
Languages vary widely in how meanings map to word forms. These mappings have been found to support efficient communication; however, this theory does not account for systematic relations within word forms. We examine how a restricted set of…
Language universals have long been attributed to an innate Universal Grammar. An alternative explanation states that linguistic universals emerged independently in every language in response to shared cognitive or perceptual biases. A…
Recent research has shown that static word embeddings can encode word frequency information. However, little has been studied about this phenomenon and its effects on downstream tasks. In the present work, we systematically study the…
Self-organization through noisy interactions is ubiquitous across physics, mathematics, and machine learning, yet how long-range structure emerges from local noisy dynamics remains poorly understood. Here, we investigate three paradigmatic…
Linguistic relations in oral conversations present how opinions are constructed and developed in a restricted time. The relations bond ideas, arguments, thoughts, and feelings, re-shape them during a speech, and finally build knowledge out…
Artificial agents have been shown to learn to communicate when needed to complete a cooperative task. Some level of language structure (e.g., compositionality) has been found in the learned communication protocols. This observed structure…
We begin with the extraordinary observation that the length distribution of 80 million proteins in UniProt, the Universal Protein Resource, measured in amino acids, is qualitatively identical to the length distribution of large collections…
With the widespread use of online social media platforms, information diffusion has become a prevalent phenomenon, making Information Diffusion Prediction (IDP) increasingly important for various applications. Despite significant…
The inverse relationship between the length of a word and the frequency of its use, first identified by G.K. Zipf in 1935, is a classic empirical law that holds across a wide range of human languages. We demonstrate that length is one…
Understanding noisy information engines is a fundamental problem of non-equilibrium physics, particularly in biomolecular systems agitated by thermal and active fluctuations in the cell. By the generalized second law of thermodynamics, the…