Related papers: Bit Rate of Programs
Signal processing and Information theory are two disparate fields used for characterizing signals for various scientific and engineering applications. Spectral/Fourier analysis, a technique employed in signal processing, helps estimation of…
This work proposes a novel computing performance unit grounded in information theory. Modern computing systems are increasingly diverse, supporting low-precision formats, hardware specialization, and emerging paradigms such as analog,…
Shannon information theory is established based on probability and bits, and the communication technology based on this theory realizes the information age. The original goal of Shannon's information theory is to describe and transmit…
Loosely speaking, the Shannon entropy rate is used to gauge a stochastic process' intrinsic randomness; the statistical complexity gives the cost of predicting the process. We calculate, for the first time, the entropy rate and statistical…
Neuroscientists formulate very different hypotheses about the nature of neural code. At one extreme, it has been argued that neurons encode information in relatively slow changes of individual spikes arriving "rates codes" and the…
Information flow analysis is a powerful technique for reasoning about the sensitive information exposed by a program during its execution. While past work has proposed information theoretic metrics (e.g., Shannon entropy, min-entropy,…
The information content of a source is defined in terms of the minimum number of bits needed to store the output of the source in a perfectly recoverable way. A similar definition can be given in the case of quantum sources, with qubits…
The learning rate is an information-theoretical quantity for bipartite Markov chains describing two coupled subsystems. It is defined as the rate at which transitions in the downstream subsystem tend to increase the mutual information…
Information theory is a statistical theory concerned with the relative state of detectors and physical systems. As a consequence, the classical framework of Shannon needs to be extended to deal with quantum detectors, possibly moving at…
In this chapter, a statistical measure of complexity and the Fisher-Shannon information product are introduced and their properties are discussed. These measures are based on the interplay between the Shannon information, or a function of…
The quantum bit error rate is a key quantity in quantum communications. If the quantum channel is the atmosphere, the information is usually encoded in the polarization of a photon. A link budget is required, which takes into account the…
This work is motivated by the need for the fundamental increase of spectral efficiency with the transmissions on the Transmission Control Protocol and the Internet Protocol (TCP/IP). To emphasize the work in physical layer, we define a…
Generation and prediction of time series is analyzed for the case of a Bit-Generator: a perceptron where in each time step the input units are shifted one bit to the right with the state of the leftmost input unit set equal to the output…
Semiconstrained systems were recently suggested as a generalization of constrained systems, commonly used in communication and data-storage applications that require certain offending subsequences be avoided. In an attempt to apply…
Quantum information is defined by applying the concepts of ordinary (Shannon) information theory to a quantum sample space consisting of a single framework or consistent family. A classical analogy for a spin-half particle and other…
Rate-distortion-perception theory generalizes Shannon's rate-distortion theory by introducing a constraint on the perceptual quality of the output. The perception constraint complements the conventional distortion constraint and aims to…
The information transmission rate (ITR), or effective bit rate, is a popular and widely used information measurement metric, particularly popularized for SSVEP-based Brain-Computer (BCI) interfaces. By combining speed and accuracy into a…
Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…
We investigate the information provided about a specified distributed apparatus of n units in the measurement of a quantum state. It is shown that, in contrast to such measurement of a classical state, which is bounded by log (n+1) bits,…
The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…