Related papers: Generalized Gibbs ensembles for time dependent pro…
Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…
Classical and quantum information theory are simply explained. To be more specific it is clarified why Shannon entropy is used as measure of classical information and after a brief review of quantum mechanics it is possible to demonstrate…
Nyquist-Shannon sampling theorem, instrumental in classical telecommunication technologies, is extended to quantum systems supporting a unitary representation of a finite group $G$. Two main ideas from the classical theory having natural…
Shannon entropy was defined for probability distributions and then its using was expanded to measure the uncertainty of knowledge for systems with complete information. In this article, it is proposed to extend the using of Shannon entropy…
Statistical equilibrium configurations are important in the physics of macroscopic systems with a large number of constituent degrees of freedom. They are expected to be crucial also in discrete quantum gravity, where dynamical spacetime…
In this paper we consider a Bayesian framework for making inferences about dynamical systems from ergodic observations. The proposed Bayesian procedure is based on the Gibbs posterior, a decision theoretic generalization of standard…
In the above mentioned paper by J. Dunkel and S. A. Trigger [Phys. Rev. {\bf A 71}, 052102, (2005)] a hypothesis has been pursued that the loss of information associated with the quantum evolution of pure states, quantified in terms of an…
In this paper we construct a theory of quantum mechanics based on Shannon information theory. We define a few principles regarding information-based frames of reference, including explicitly the concept of information covariance, and show…
Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic…
In information theory, one major goal is to find useful functions that summarize the amount of information contained in the interaction of several random variables. Specifically, one can ask how the classical Shannon entropy, mutual…
We derive fluctuation relations for a many-body quantum system prepared in a Generalised Gibbs Ensemble subject to a general nonequilibrium protocol. By considering isolated integrable systems, we find generalisations to the Tasaki-Crooks…
We revisit the work of Mitter and Newton on an information-theoretic interpretation of Bayes' formula through the Gibbs variational principle. This formulation allowed them to pose nonlinear estimation for diffusion processes as a problem…
All natural things process and transform information. They receive environmental information as input, and transform it into appropriate output responses. Much of science is dedicated to building models of such systems -- algorithmic…
For a quantum system in a macroscopically large volume $V$, prepared in a pure state and subject to maximally noisy or ergodic unitary dynamics, the reduced density matrix of any sub-system $v\ll V$ is almost surely totally mixed. We show…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
It is argued that a Gibbsian formula for the space-time distribution of microscopic trajectories of a nonequilibrium system provides a unifying framework for recent results on the fluctuations of the entropy production. The variable entropy…
We investigate the overdamped stochastic dynamics of a particle in an asymptotically flat external potential field, in contact with a thermal bath. For an infinite system size, the particles may escape the force field and diffuse freely at…
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…
The rapid scaling of artificial intelligence models has revealed a fundamental tension between model capacity (storage) and inference efficiency (computation). While classical information theory focuses on transmission and storage limits,…
We analyze dynamical properties of the Shannon information entropy of a continuous probability distribution, which is driven by a standard diffusion process. This entropy choice is confronted with another option, employing the conditional…