Related papers: The Entropy of Parallel Systems
Transfer entropy is capable of capturing nonlinear source-destination relations between multi-variate time series. It is a measure of association between source data that are transformed into destination data via a set of linear…
Understanding the limitations imposed by noise on current and next-generation quantum devices is a crucial step towards demonstrating practical quantum advantage. In this work, we investigate the accumulation of entropy density as a…
Highly excited many-particle states in quantum systems such as nuclei, atoms, quantum dots, spin systems, quantum computers etc., can be considered as ``chaotic'' superpositions of mean-field basis states (Slater determinants, products of…
The average uncertainty associated with words is an information-theoretic concept at the heart of quantitative and computational linguistics. The entropy has been established as a measure of this average uncertainty - also called average…
Ensemble of initial conditions for nonlinear maps can be described in terms of entropy. This ensemble entropy shows an asymptotic linear growth with rate K. The rate K matches the logarithm of the corresponding asymptotic sensitivity to…
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…
Neural networks achieve remarkable performance through superposition: encoding multiple features as overlapping directions in activation space rather than dedicating individual neurons to each feature. This challenges interpretability, yet…
The notion of software entropy is often invoked to describe the tendency of software systems to become increasingly disordered as they evolve, yet existing approaches to quantify it are largely heuristic. In this work we introduce a formal…
We use entropy-energy arguments to assess the limitations on the running time and on the system size, as measured in qubits, of noisy macroscopic circuit-based quantum computers.
Whether noisy quantum devices without error correction can provide quantum advantage over classical computers is a critical issue of current quantum computation. In this work, the random quantum circuits, which are used as the paradigm…
We introduce a new methodology for the analysis of the phenomenon of chaotic itinerancy in a dynamical system using the notion of entropy and a clustering algorithm. We determine systems likely to experience chaotic itinerancy by means of…
Some quantum algorithms have "quantum speedups": improved time complexity as compared with the best-known classical algorithms for solving the same tasks. Can we understand what fuels these speedups from an entropic perspective? Information…
We explore the connection between deep learning and information theory through the paradigm of diffusion models. A diffusion model converts noise into structured data by reinstating, imperfectly, information that is erased when data was…
Quantum informatic quantities such as entanglement entropy are useful in detecting quantum phase transitions. Recently, a new entanglement measure called pseudo-entropy was proposed which is a generalization of the more well-known…
The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…
We introduce a simple, yet novel entropy-based framework to drive token efficiency in large language models during reasoning tasks. Our approach uses Shannon entropy from token-level logprobs as a confidence signal to enable early stopping,…
Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building…
In this paper, we introduce the notion of recurrence entropy in the context of nonlinear control systems. A set is said to be ($\tau$-)recurrent if every trajectory that starts in the set returns to it (within at most $\tau$ units of time).…
In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…
Fluctuations in parameters that are typically treated as fixed play a crucial role in the behavior of complex systems. However, to date, we lack a general non-equilibrium thermodynamic treatment of such a complex system. In this Letter, to…