Related papers: Towards the full information chain theory: questio…
Information theory is an outstanding framework to measure uncertainty, dependence and relevance in data and systems. It has several desirable properties for real world applications: it naturally deals with multivariate data, it can handle…
We present an information-theoretic interpretation of quantum formalism based on a Bayesian framework and devoid of any extra axiom or principle. Quantum information is construed as a technique for analyzing a logical system subject to…
Interpretability is a pressing issue for machine learning. Common approaches to interpretable machine learning constrain interactions between features of the input, rendering the effects of those features on a model's output comprehensible…
We introduce a hierarchical classification of theories that describe systems with fundamentally limited information content. This property is introduced in an operational way and gives rise to the existence of mutually complementary…
Constraints on entropies are considered to be the laws of information theory. Even though the pursuit of their discovery has been a central theme of research in information theory, the algorithmic aspects of constraints on entropies remain…
The significance of statistical physics concepts such as entropy extends far beyond classical thermodynamics. We interpret the similarity between partitions in statistical mechanics and partitions in Bayesian inference as an articulation of…
We develop a general formalism for representing and understanding structure in complex systems. In our view, structure is the totality of relationships among a system's components, and these relationships can be quantified using information…
Information inequalities appear in many database applications such as query output size bounds, query containment, and implication between data dependencies. Recently Khamis et al. proposed to study the algorithmic aspects of information…
The study of intelligent systems explains behaviour in terms of economic rationality. This results in an optimization principle involving a function or utility, which states that the system will evolve until the configuration of maximum…
Three-way decision is widely applied with rough set theory to learn classification or decision rules. The approaches dealing with complete information are well established in the literature, including the two complementary computational and…
Information-theoretic quantities, such as entropy, are used to quantify the amount of information a given variable provides. Entropies can be used together to compute the mutual information, which quantifies the amount of information two…
How best to quantify the information of an object, whether natural or artifact, is a problem of wide interest. A related problem is the computability of an object. We present practical examples of a new way to address this problem. By…
In inverse problems, one attempts to infer spatially variable functions from indirect measurements of a system. To practitioners of inverse problems, the concept of "information" is familiar when discussing key questions such as which parts…
In our former work [K. Tadaki, Local Proceedings of CiE 2008, pp.425-434, 2008], we developed a statistical mechanical interpretation of algorithmic information theory by introducing the notion of thermodynamic quantities at temperature T,…
We investigate the fundamental limitations imposed by thermodynamics for creating correlations. Considering a collection of initially uncorrelated thermal quantum systems, we ask how much classical and quantum correlations can be obtained…
The necessary information for specifying a complex system may not be completely accessible to us, i.e., to mathematical treatments. This is not to be confounded with the incompleteness of our knowledge about whatever systems or nature,…
Building on the fundamental equation, this study revisits key thermodynamic concepts in a cohesive and innovative manner. It demonstrates the consistency of thermodynamic theory while addressing and clarifying common misconceptions and…
Information theory provides tools to predict the performance of a learning algorithm on a given dataset. For instance, the accuracy of learning an unknown parameter can be upper bounded by reducing the learning task to hypothesis testing…
Quantum information theory is the study of the achievable limits of information processing within quantum mechanics. Many different types of information can be accommodated within quantum mechanics, including classical information, coherent…
The process of creating goods and services, measured by their value, is considered a process of creating complexity. This allows us to consider the production system as an open thermodynamic system, and to develop a simple heuristic model…