相关论文: Maximum Entropy Rate of Markov Sources for Systems…

Shannon's Capacity Theorem is the main concept behind the Theory of Communication. It says that if the amount of information contained in a signal is smaller than the channel capacity of a physical media of communication, it can be…

Every channel can be expressed as a convex combination of deterministic channels with each deterministic channel corresponding to one particular intrinsic state. Such convex combinations are in general not unique, each giving rise to a…

MaxEnt's variational principle, in conjunction with Shannon's logarithmic information measure, yields only exponential functional forms in straightforward fashion. In this communication we show how to overcome this limitation via the…

New non-asymptotic random coding theorems (with error probability $\epsilon$ and finite block length $n$) based on Gallager parity check ensemble and Shannon random code ensemble with a fixed codeword type are established for discrete input…

We study reliable communication in uncoordinated vehicular communication from the perspective of Shannon theory. Our system model for the information transmission is that of an Arbitrarily Varying Channel (AVC): One sender-receiver pair…

There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the…

As an application of generalised statistical mechanics, it is studied a possible route toward a consistent generalised information theory in terms of a family of non-extensive, non-parametric entropies $H^\pm_D(P)$. Unlike other proposals…

Information theory is introduced in this lecture note with a particular emphasis on its relevance to algebraic coding theory. The document develops the mathematical foundations for quantifying uncertainty and information transmission by…

We consider the problem of covert communication with random slot selection over binary-input Discrete Memoryless Channels and Additive White Gaussian Noise channels, in which a transmitter attempts to reliably communicate with a legitimate…

Evaluating the channel capacity is one of many key problems in information theory. In this work we derive rather-mild sufficient conditions under which the capacity is finite and achievable. These conditions are derived for generic,…

A random chaotic interval map with noise which causes coarse-graining induces a finite-state Markov chain. For a map topologically conjugate to a piecewise-linear map with the Lebesgue measure being ergodic, we prove that the Shannon…

Two familiar notions of correlation are rediscovered as the extreme operating points for distributed synthesis of a discrete memoryless channel, in which a stochastic channel output is generated based on a compressed description of the…

Empirical coordination offers a way to understand how agents can coordinate actions under communication constraints. This paper investigates the finite blocklength regime of this problem, where the encoder and decoder aim to produce a…

We extend the notion of estimation entropy of autonomous dynamical systems proposed by Liberzon and Mitra [1] to nonlinear dynamical systems with uncertain inputs with bounded variation. We call this new notion the {$\epsilon$}-estimation…

Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…

This paper describes a partial parser that assigns syntactic structures to sequences of part-of-speech tags. The program uses the maximum entropy parameter estimation method, which allows a flexible combination of different knowledge…

In this correspondence, we illustrate among other things the use of the stationarity property of the set of capacity-achieving inputs in capacity calculations. In particular, as a case study, we consider a bit-patterned media recording…

We introduce a resource theory of channels relevant to communication via quantum channels, in which the set of constant channels --- useless channels for communication tasks --- is considered as the free resource. We find that our theory…

The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…

Consider communication over a channel whose probabilistic model is completely unknown vector-wise and is not assumed to be stationary. Communication over such channels is challenging because knowing the past does not indicate anything about…