Related papers: Information without rolling dice
We investigate how undecidability enters into computations of classical physical systems and contributes to the increase of entropy and loss of information. In actual computation with finite bit of information capacity we accept…
Evaluating the channel capacity is one of many key problems in information theory. In this work we derive rather-mild sufficient conditions under which the capacity is finite and achievable. These conditions are derived for generic,…
With the recent development of quantum information theory, some attempts exist to construct information theory beyond quantum theory. Here we consider hypothesis testing relative entropy and one-shot classical capacity, that is, the optimal…
We investigate state estimation of linear systems over channels having a finite state not known by the transmitter or receiver. We show that similar to memoryless channels, zero-error capacity is the right figure of merit for achieving…
This paper considers the problem of stabilizing a discrete-time non-linear stochastic system over a finite capacity noiseless channel. Our focus is on systems which decompose into a stable and unstable component, and the stability notion…
The information-theoretic representation of quantum systems, which complements the familiar energy description of the density-functional and wave-function-based theories, is here discussed. According to it, the internal disorder of the…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
It is not obvious how to extend Shannon's original information entropy to higher dimensions, and many different approaches have been tried. We replace the English text symbol sequence originally used to illustrate the theory by a discrete,…
The design of error-correcting codes used in modern communications relies on information theory to quantify the capacity of a noisy channel to send information [1]. This capacity can be expressed using the mutual information between input…
In this work, conditional entropy is used to quantify the information loss induced by passing a continuous random variable through a memoryless nonlinear input-output system. We derive an expression for the information loss depending on the…
Our capacity to process information depends on the computational power at our disposal. Information theory captures our ability to distinguish states or communicate messages when it is unconstrained with unrivaled beauty and elegance. For…
Information theory establishes the fundamental limits on data transmission, storage, and processing. Quantum information theory unites information theoretic ideas with an accurate quantum-mechanical description of reality to give a more…
The problems of causality, modeling, and control for chaotic, high-dimensional dynamical systems are formulated in the language of information theory. The central quantity of interest is the Shannon entropy, which measures the amount of…
We use the formalism of 'Maximum Principle of Shannon's Entropy' to derive the general power law distribution function, using what seems to be a reasonable physical assumption, namely, the demand of a constant mean "internal order"…
The information entropy budget and the rate of information transfer between variables is studied in the context of a nonlinear reduced-order atmospheric model. The key ingredients of the dynamics are present in this model, namely the…
The rapid scaling of artificial intelligence models has revealed a fundamental tension between model capacity (storage) and inference efficiency (computation). While classical information theory focuses on transmission and storage limits,…
We present a quantum information theory that allows for the consistent description of quantum entanglement. It parallels classical (Shannon) information theory but is based entirely on density matrices, rather than probability…
A central concept in the connection between physics and information theory is entropy, which represents the amount of information extracted from the system by the observer performing measurements in an experiment. Indeed, Jaynes' principle…
The entanglement-assisted classical capacity of a noisy quantum channel is the amount of information per channel use that can be sent over the channel in the limit of many uses of the channel, assuming that the sender and receiver have…
In information theory, one major goal is to find useful functions that summarize the amount of information contained in the interaction of several random variables. Specifically, one can ask how the classical Shannon entropy, mutual…