Related papers: Measuring Knowledge for Recognition and Knowledge …
Human knowledge provides a formal understanding of the world. Knowledge graphs that represent structural relations between entities have become an increasingly popular research direction towards cognition and human-level intelligence. In…
This paper develops an inconsistency measure on conditional probabilistic knowledge bases. The measure is based on fundamental principles for inconsistency measures and thus provides a solid theoretical framework for the treatment of…
As a part of the construction of an information theory based on general probabilistic theories, we propose and investigate the several distinguishability measures and "entropies" in general probabilistic theories. As their applications,…
Problem of classification of all the set of entangled states is considered. Invariance of entangled states relative to transformations from a group of symmetry of qubit space leads to classification of all states of the system through…
Calibration is a popular framework to evaluate whether a classifier knows when it does not know - i.e., its predictive probabilities are a good indication of how likely a prediction is to be correct. Correctness is commonly estimated…
In this work, we apply information theory inspired methods to quantify changes in daily activity patterns. We use in-home movement monitoring data and show how they can help indicate the occurrence of healthcare-related events. Three…
A novel, non-trivial, probabilistic upper bound on the entropy of an unknown one-dimensional distribution, given the support of the distribution and a sample from that distribution, is presented. No knowledge beyond the support of the…
This paper presents a method of understanding the growth of global science as resulting from a mechanism of preferential attachment within networks. The paper seeks to contribute to the development of indicators of knowledge creation and…
By 'informational entropy', we understand an inherent boundary to knowability, due e.g. to perceptual, theoretical, evidential or linguistic limits. In this paper, we discuss a logical framework in which this boundary is incorporated into…
We begin with the premise that the law of entropy could prove to be fundamental for the evolution of intelligent life and the advent of technological civilization. Building on recent theoretical results, we combine a modern approach to…
While large language models (LLMs) demonstrate strong capabilities across diverse user queries, they still suffer from hallucinations, often arising from knowledge misalignment between pre-training and fine-tuning. To address this…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…
These lecture notes concern information-theoretic notions of entropy. They are intended for, and have been successfully taught to, undergraduate students interested inresearch careers. Besides basic notions of analysis related to…
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
We study distributed knowledge, which is what privately informed agents come to know by communicating freely with one another and sharing everything they know. Knowledge is not necessarily partitional: agents may be boundedly rational and…
When a robot learns from human examples, most approaches assume that the human partner provides examples of optimal behavior. However, there are applications in which the robot learns from non-expert humans. We argue that the robot should…
We study an entropy measure for quantum systems that generalizes the von Neumann entropy as well as its classical counterpart, the Gibbs or Shannon entropy. The entropy measure is based on hypothesis testing and has an elegant formulation…
Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, $S=-\sum_i p_i \log p_i$. Here we note that in physical systems one of these axioms may be violated. For non-ergodic…
To provide an explanation of the evolution of scientific knowledge, I start from the assumption that knowledge is based on concepts, and propose that each concept about reality is affected by vagueness. This entails a paradox, which I term…
Shannon's entropy and other entropy-based concepts are derived from the new, more general concept of relative divergence of one "grading' function on a linearly ordered set from another such function. The definition of relative divergence…