Related papers: Non-Stochastic Information Theory
This Thesis explores how tools from Statistical Physics and Information Theory can help us describe and understand complex systems. In the first part, we study the interplay between internal interactions, environmental changes, and…
The data for many classification problems, such as pattern and speech recognition, follow mixture distributions. To quantify the optimum performance for classification tasks, the Shannon mutual information is a natural information-theoretic…
We introduce a resource theory of channels relevant to communication via quantum channels, in which the set of constant channels --- useless channels for communication tasks --- is considered as the free resource. We find that our theory…
Model selection in clustering requires (i) to specify a suitable clustering principle and (ii) to control the model order complexity by choosing an appropriate number of clusters depending on the noise level in the data. We advocate an…
One of the primary goals of information theory is to provide limits on the amount of information it is possible to send through various types of communication channels, and to understand the encoding methods that will allow one to achieve…
The tails of prehistory probability density in nonlinear multistable stochastic systems driven by white Gaussian noise, which has been a subject of recent study, are analyzed by employing the concepts of nonstationary optimal fluctuations.…
We propose a general framework, dubbed Stochastic Processing under Imperfect Information (SPII), to study the impact of information constraints and memories on dynamic resource allocation. The framework involves a Stochastic Processing…
Shannon's analysis of the fundamental capacity limits for memoryless communication channels has been refined over time. In this paper, the maximum volume $M_\avg^*(n,\epsilon)$ of length-$n$ codes subject to an average decoding error…
We consider a general stochastic input-output dynamical system with output evolving in time as the solution to a functional coefficients, It\^{o}'s stochastic differential equation, excited by an input process. This general class of…
In classical information theory, both the form and performance of the optimal detector for additive noise channels can be precisely derived, based on the assumption that the channel noise follows a specific probability distribution or a…
This paper considers the problem of stabilizing a discrete-time non-linear stochastic system over a finite capacity noiseless channel. Our focus is on systems which decompose into a stable and unstable component, and the stability notion…
The Shannon upper bound places a limit on the error-free information transmission rate (capacity) of a noisy channel. It has stood for over sixty years, and underlies both theoretical and practical work in the telecommunications industry.…
Non-asymptotic quantum Shannon theory analyses how to transmit quantum information across a quantum channel as efficiently as possible within a specified error tolerance, given access to a finite, fixed, number of channel uses. In a recent…
The concept of stochastic resonance in nonlinear dynamics is applied to interpret the capacity of noisy quantum channels. The two-Pauli channel is used to illustrate the idea. The fidelity of the channel is also considered. Noise…
This work considers a Poisson noise channel with an amplitude constraint. It is well-known that the capacity-achieving input distribution for this channel is discrete with finitely many points. We sharpen this result by introducing upper…
It is proved that the width of a function and the width of the distribution of its values cannot be made arbitrarily small simultaneously. In the case of ergodic stochastic processes, an ensuing uncertainty relationship is demonstrated for…
Quantum technologies rely on the ability to coherently manipulate, process and transfer information, encoded in quantum states, along quantum channels. Decoherence induced by the environment introduces errors, thus setting limits on the…
From the output produced by a memoryless deletion channel with a uniformly random input of known length $n$, one obtains a posterior distribution on the channel input. The difference between the Shannon entropy of this distribution and that…
Memoryless channels with deletion errors as defined by a stochastic channel matrix allowing for bit drop outs are considered in which transmitted bits are either independently deleted with probability $d$ or unchanged with probability…
Any physical channel of communication offers two potential reasons why its capacity (the number of bits it can transmit in a unit of time) might be unbounded: (1) Infinitely many choices of signal strength at any given instant of time, and…