Related papers: Quantifying information via Shannon entropy in spa…
The system identification capabilities of a novel information-theoretic method are examined here. Specifically, this work uses information-theoretic metrics and vibration-based measurements to enhance damping estimation accuracy in…
Information plays an important role in our understanding of the physical world. We hence propose an entropic measure of information for any physical theory that admits systems, states and measurements. In the quantum and classical world,…
This paper revisits Brownian motion from the perspective of Information Theory, aiming to explore the connections between Information Theory, Thermodynamics, and Complex Science. First, we propose a single-particle discrete Brownian motion…
Under the constraint of constant illumination, an information criterion is formulated for the Fisher information that compressed sensing measurements in optical and transmission electron microscopy contain about the underlying parameters.…
Rare events attract more attention and interests in many scenarios of big data such as anomaly detection and security systems. To characterize the rare events importance from probabilistic perspective, the message importance measure (MIM)…
We consider the rate-distortion function for lossy source compression, as well as the channel capacity for error correction, through the lens of distributional robustness. We assume that the distribution of the source or of the additive…
Complex adaptive systems (CAS) can be described as systems of information flows dynamically interacting across scales in order to adapt and survive. CAS often consist of many components that work towards a shared goal, and interact across…
Several variations of the Heisenberg uncertainty inequality are derived on the basis of "noise-resolution duality" recently proposed by the authors. The same approach leads to a related inequality that provides an upper limit for the…
An information theory description of finite systems explicitly evolving in time is presented for classical as well as quantum mechanics. We impose a variational principle on the Shannon entropy at a given time while the constraints are set…
We survey and introduce concepts and tools located at the intersection of information theory and network biology. We show that Shannon's information entropy, compressibility and algorithmic complexity quantify different local and global…
Information entropic measures such as Fisher information, Shannon entropy, Onicescu energy and Onicescu Shannon entropy of a symmetric double-well potential are calculated in both position and momentum space. Eigenvalues and eigenvectors of…
We define a notion of information that an individual sample provides to the training of a neural network, and we specialize it to measure both how much a sample informs the final weights and how much it informs the function computed by the…
We consider the analysis of probability distributions through their associated covariance operators from reproducing kernel Hilbert spaces. We show that the von Neumann entropy and relative entropy of these operators are intimately related…
A new generalized Statistical Complexity Measure (SCM) was proposed by Rosso et al in 2010. It is a functional that captures the notions of order/disorder and of distance to an equilibrium distribution. The former is computed by a measure…
For a certain moment, the information volume represented in a probability space can be accurately measured by Shannon entropy. But in real life, the results of things usually change over time, and the prediction of the information volume…
We provide a stochastic extension of the Baez-Fritz-Leinster characterization of the Shannon information loss associated with a measure-preserving function. This recovers the conditional entropy and a closely related information-theoretic…
Within the expansive domain of optical sciences, achieving the precise characterization of light beams stands as a fundamental pursuit, pivotal for various applications, including telecommunications and imaging technologies. This study…
Optical phase-modulated fringe-patterns are usually digitized with XxY pixels and 8 bits/pixel (or higher) gray-levels. The digitized 8 bits/pixel are raw-data bits, not Shannon information bits. Here we show that noisy fringe-patterns…
Signal-to-Noise Ratios (SNRs) and the Shannon-Hartley channel capacity are metrics that help define the loss of known information while transferring data through a noisy channel. These metrics cannot be used for quantifying the opposite…
The body morphology plays an important role in the way information is perceived and processed by an agent. We address an information theory (IT) account on how the precision of sensors, the accuracy of motors, their placement, the body…