Related papers: On how generalised entropies without parameters im…
Living cell signaling systems include multistep biochemical signaling reaction cascades (BSCs) comprising modifications of molecular signaling proteins. Substantial data on BSCs have been accumulated in the field of molecular biology and…
An information theory description of finite systems explicitly evolving in time is presented for classical as well as quantum mechanics. We impose a variational principle on the Shannon entropy at a given time while the constraints are set…
Anyonic systems are modeled by topologically protected Hilbert spaces which obey complex superselection rules restricting possible operations. These Hilbert spaces cannot be decomposed into tensor products of spatially localized subsystems,…
Shannon's channel coding theorem characterizes the maximal rate of information that can be reliably transmitted over a communication channel when optimal encoding and decoding strategies are used. In many scenarios, however, practical…
Nature is full of random networks of complex topology describing such apparently disparate systems as biological, economical or informatical ones. Their most characteristic feature is the apparent scale-free character of interconnections…
Binary concepts are empirically used by humans to generalize efficiently. And they are based on Bernoulli distribution which is the building block of information. These concepts span both low-level and high-level features such as "large vs…
Bounds on information combining are entropic inequalities that determine how the information, or entropy, of a set of random variables can change when they are combined in certain prescribed ways. Such bounds play an important role in…
A general method of coding over expansion is proposed,which allows one to reduce the highly non-trivial problems of coding over analog channels and compressing analog sources to a set of much simpler subproblems, coding over discrete…
Language prediction is constrained by informational entropy intrinsic to language, such that there exists a limit to how accurate any language model can become and equivalently a lower bound to language compression. The most efficient…
As one of the potential key technologies of 6G, semantic communication is still in its infancy and there are many open problems, such as semantic entropy definition and semantic channel coding theory. To address these challenges, we…
This paper develops an envelope-based approach to establish a link between information and queueing theory. Unlike classical, equilibrium information theory, information envelopes focus on the dynamics of sources and coders, using functions…
We propose a new interpretation of measures of information and disorder by connecting these concepts to group theory in a new way. Entropy and group theory are connected here by their common relation to sets of permutations. A combinatorial…
We investigate state estimation of linear systems over channels having a finite state not known by the transmitter or receiver. We show that similar to memoryless channels, zero-error capacity is the right figure of merit for achieving…
The rapidly improving performance of modern hardware renders convolutional codes obsolete, and allows for the practical implementation of more sophisticated correction codes such as low density parity check (LDPC) and turbo codes (TC). Both…
Whereas Shannon entropy is related to the growth rate of multinomial coefficients, we show that the quadratic entropy (Tsallis 2-entropy) is connected to their $q$-deformation; when $q$ is a prime power, these $q$-multinomial coefficients…
Statistical mechanics is generalized on the basis of an additive information theory for incomplete probability distributions. The incomplete normalization $\sum_{i=1}^wp_i^q=1$ is used to obtain generalized entropy $S=-k\sum_{i=1}^wp_i^q\ln…
Hierarchical structures, which include multiple levels, are prevalent in statistical and machine-learning models as well as physical systems. Extending the foundational result that the maximum entropy distribution under mean constraints is…
We first show how a new definition of entropy, which is intuitively very simple, as a divergence in cluster-size space, leads to a generalized form that is nonextensive for correlated units, but coincides exactly with the conventional one…
Analysis of a probabilistic system often requires to learn the joint probability distribution of its random variables. The computation of the exact distribution is usually an exhaustive precise analysis on all executions of the system. To…
This work proposes lossless and near-lossless compression algorithms for multi-channel biomedical signals. The algorithms are sequential and efficient, which makes them suitable for low-latency and low-power signal transmission…