Related papers: Criticality in Formal Languages and Statistical Ph…
In artificial neural networks, the activation dynamics of non-trainable variables is strongly coupled to the learning dynamics of trainable variables. During the activation pass, the boundary neurons (e.g., input neurons) are mapped to the…
Understanding texts requires memory: the reader has to keep in mind enough words to create meaning. This calls for a relation between the memory of the reader and the structure of the text. To investigate this interaction, we first identify…
Zipf's law predicts a power-law relationship between word rank and frequency in language communication systems and has been widely reported in a variety of natural language processing applications. However, the emergence of natural language…
Language models must capture statistical dependencies between words at timescales ranging from very short to very long. Earlier work has demonstrated that dependencies in natural language tend to decay with distance between words according…
We studied the mutual information between a stimulus and a large system consisting of stochastic, statistically independent elements that respond to a stimulus. The Mutual Information (MI) of the system saturates exponentially with system…
Does information play a significant role in the foundations of physics? Information is the abstraction that allows us to refer to the states of systems when we choose to ignore the systems themselves. This is only possible in very…
Recent theories suggest that Neural Scaling Laws arise whenever the task is linearly decomposed into power-law distributed units. Alternatively, scaling laws also emerge when data exhibit a hierarchically compositional structure, as is…
The influence of additional information on the decision making of agents, who are interacting members of a society, is analyzed within the mathematical framework based on the use of quantum probabilities. The introduction of social…
The two-body potential of systems with long-range interactions decays at large distances as $V(r)\sim 1/r^\alpha$, with $\alpha\leq d$, where $d$ is the space dimension. Examples are: gravitational systems, two-dimensional hydrodynamics,…
We investigate a two-qubit system to understand the relationship between concurrence and mutual information, where the former determines the amount of quantum entanglement, whereas the latter is its classical residue after performing local…
We consider the estimation of a signal from the knowledge of its noisy linear random Gaussian projections, a problem relevant in compressed sensing, sparse superposition codes or code division multiple access just to cite few. There has…
Temporal information conveyed by language describes how the world around us changes through time. Events, durations and times are all temporal elements that can be viewed as intervals. These intervals are sometimes temporally related in…
We investigate mechanisms for language change within a framework where an unconventional signal for a meaning is first innovated, and then subsequently propagated through a speech community to replace the existing convention. We appeal to…
Correlations disguised in various forms underlie a host of important phenomena in classical and quantum systems, such as information and energy exchanges. The quantum mutual information and the norm of the correlation matrix are both…
We perform a numerical study of the long range (LR) ferromagnetic Ising model with power law decaying interactions ($J \propto r^{-d-\sigma}$) both on a one-dimensional chain ($d=1$) and on a square lattice ($d=2$). We use advanced cluster…
Intuitively, the more powerful a theory is, the greater the variety and quantity of ideas can be expressed through its formal language. Therefore, when comparing two theories concerning the same subject, it seems only reasonable to compare…
This is an ongoing review on the idea that the phase space information loss at causal horizons is the key ingredient of major physical laws. Assuming that information is fundamental and the information propagates with finite velocity, one…
Statistical mechanical concepts and processes such as decoherence, correlation, and dissipation can prove to be of basic importance to understanding some fundamental issues of quantum cosmology and theoretical physics such as the choice of…
The strong coupling limit of a quantum system is in general quite complicated, but in some cases a great simplification occurs: the strongly coupled limit is equivalent to the weakly coupled limit of some other system. In string theory…
The longstanding question of how stochastic behaviour arises from deterministic Hamiltonian dynamics is of great importance, and any truly holistic theory must be capable of describing this transition. In this review, we introduce the…