相关论文: On the origin of ambiguity in efficient communicat…

In many biological networks the responses of individual elements are ambiguous. We consider a scenario in which many sensors respond to a shared signal, each with limited information capacity, and ask that the outputs together convey as…

One of the key aspects of Shannon's theory is that it provides guidance for designing the most efficient systems, such as minimizing errors and clarifying the limits of coding. Such theories have made great developments in the 50 years…

We show why the amount of information communicated between the past and future--the excess entropy--is not in general the amount of information stored in the present--the statistical complexity. This is a puzzle, and a long-standing one,…

This article introduces the physics of information in the context of molecular biology and genomics. Entropy and information, the two central concepts of Shannon's theory of information and communication, are often confused with each other…

Suppose that Alice and Bob are given each an infinite string, and they want to decide whether their two strings are in a given relation. How much communication do they need? How can communication be even defined and measured for infinite…

The construction of a consistent theory for structuring and representing how concepts combine and interact is one of the main challenges for the scholars involved in cognitive studies. All traditional approaches are still facing serious…

Zipf's law has been found in many human-related fields, including language, where the frequency of a word is persistently found as a power law function of its frequency rank, known as Zipf's law. However, there is much dispute whether it is…

The article presents a new interpretation for Zipf-Mandelbrot's law in natural language which rests on two areas of information theory. Firstly, we construct a new class of grammar-based codes and, secondly, we investigate properties of…

Algorithms of inference in a computer system oriented to input and semantic processing of text information are presented. Such inference is necessary for logical questions when the direct comparison of objects from a question and database…

Claude Shannons information theory (1949) has had a revolutionary impact on communication science. A crucial property of his framework is that it decouples the meaning of a message from the mechanistic details from the actual communication…

Reversible computation is key in developing new, energy-efficient paradigms, but also in providing forward-only concepts with broader definitions and finer frames of study.Among other fields, the algebraic specification and representation…

Natural language processing has made significant inroads into learning the semantics of words through distributional approaches, however representations learnt via these methods fail to capture certain kinds of information implicit in the…

Question processing is a fundamental step in a question answering (QA) application, and its quality impacts the performance of QA application. The major challenging issue in processing question is how to extract semantic of natural language…

Wireless communication has achieved great success in the past several decades. The challenge is of improving bandwidth with limited spectrum and power consumption, which however has gradually become a bottleneck with evolution going on. The…

Despite the frequent challenges posed by ambiguity when representing meaning via natural language, it is often ignored or deliberately removed in tasks mapping language to formally-designed representations, which generally assume a…

The nature of concept learning is a core question in cognitive science. Theories must account for the relative difficulty of acquiring different concepts by supervised learners. For a canonical set of six category types, two distinct…

In many complex systems, whether biological or artificial, the thermodynamic costs of communication among their components are large. These systems also tend to split information transmitted between any two components across multiple…

Semantic communication is envisioned as a promising technique to break through the Shannon limit. However, the existing semantic communication frameworks do not involve inference and error correction, which limits the achievable…

Artificial intelligence and machine learning algorithms have become ubiquitous. Although they offer a wide range of benefits, their adoption in decision-critical fields is limited by their lack of interpretability, particularly with textual…

A simple framework for reasoning under uncertainty and intervention is introduced. This is achieved in three steps. First, logic is restated in set-theoretic terms to obtain a framework for reasoning under certainty. Second, this framework…