相关论文: Shannon Shakes Hands with Chernoff: Big Data Viewp…
A measurement performed on a quantum system is an act of gaining information about its state, a view that is widespread in practical and foundational work in quantum theory. However, the concept of information in quantum theory…
In Shannon's seminal paper, entropy of printed English, treated as a stationary stochastic process, was estimated to be roughly 1 bit per character. However, considered as a means of communication, language differs considerably from its…
In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…
Shannon's Capacity Theorem is the main concept behind the Theory of Communication. It says that if the amount of information contained in a signal is smaller than the channel capacity of a physical media of communication, it can be…
The paper examines relationships between the Shannon entropy and the $\ell_{\alpha}$-norm for $n$-ary probability vectors, $n \ge 2$. More precisely, we investigate the tight bounds of the $\ell_{\alpha}$-norm with a fixed Shannon entropy,…
The Shannon entropy is a widely used summary statistic, for example, network traffic measurement, anomaly detection, neural computations, spike trains, etc. This study focuses on estimating Shannon entropy of data streams. It is known that…
Entropic measures provide analytic tools to help us understand correlation in quantum systems. In our previous work, we calculated linear entropy and von Neumann entropy as entanglement measures for the ground state and lower lying excited…
Change detection within an audio stream is an important task in several domains, such as classification and segmentation of a sound or of a music piece, as well as indexing of broadcast news or surveillance applications. In this paper we…
Shannon's fundamental bound for perfect secrecy says that the entropy of the secret message cannot be larger than the entropy of the secret key initially shared by the sender and the legitimate receiver. Massey gave an information theoretic…
An identity between two versions of the Chernoff bound on the probability a certain large deviations event, is established. This identity has an interpretation in statistical physics, namely, an isothermal equilibrium of a composite system…
Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…
Shannon's secrecy system is studied in a setting, where both the legitimate decoder and the wiretapper have access to side information sequences correlated to the source, but the wiretapper receives both the coded information and the side…
Complex adaptive systems (CAS) can be described as systems of information flows dynamically interacting across scales in order to adapt and survive. CAS often consist of many components that work towards a shared goal, and interact across…
With the aid of a toy model, the Monty Hall Problem (MHP), the counterintuitive and theoretically problematic concept of negative information in classical systems is well understood. It is shown that, as its quantum counterpart, classical…
Fano's inequality is one of the most elementary, ubiquitous, and important tools in information theory. Using majorization theory, Fano's inequality is generalized to a broad class of information measures, which contains those of Shannon…
We consider the Shannon mutual information of subsystems of critical quantum chains in their ground states. Our results indicate a universal leading behavior for large subsystem sizes. Moreover, as happens with the entanglement entropy, its…
Shannon mutual information provides a measure of how much information is, on average, contained in a set of neural activities about a set of stimuli. It has been extensively used to study neural coding in different brain areas. To apply a…
As an application of generalised statistical mechanics, it is studied a possible route toward a consistent generalised information theory in terms of a family of non-extensive, non-parametric entropies $H^\pm_D(P)$. Unlike other proposals…
Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability…
This paper introduces time into information theory, gives a more accurate definition of information, and unifies the information in cognition and Shannon information theory. Specially, we consider time as a measure of information, giving a…