Related papers: Lectures on Probability, Entropy, and Statistical …
Although some information-theoretic measures of uncertainty or granularity have been proposed in rough set theory, these measures are only dependent on the underlying partition and the cardinality of the universe, independent of the lower…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
We consider the problem of diagnosis where a set of simple observations are used to infer a potentially complex hidden hypothesis. Finding the optimal subset of observations is intractable in general, thus we focus on the problem of active…
Making decisions freely presupposes that there is some indeterminacy in the environment and in the decision making engine. The former is reflected on the behavioral changes due to communicating: few changes indicate rigid environments;…
In quantum mechanics, outcomes of measurements on a state have a probabilistic interpretation while the evolution of the state is treated deterministically. Here we show that one can also treat the evolution as being probabilistic in nature…
This paper examines some methods and ideas underlying the author's successful probabilistic learning systems(PLS), which have proven uniquely effective and efficient in generalization learning or induction. While the emerging principles are…
Entropy is one of the key thermodynamic variables reflecting changes in the state of matter. Unlike other thermodynamic variables, it is well-defined also for nonequilibrium steady states through its relation to information. Applying this…
We introduce a class of information measures based on group entropies, allowing us to describe the information-theoretical properties of complex systems. These entropic measures are nonadditive, and are mathematically deduced from a series…
Various strategies for active learning have been proposed in the machine learning literature. In uncertainty sampling, which is among the most popular approaches, the active learner sequentially queries the label of those instances for…
Starting from considerations about meaning and subsequent use of asymmetric uncertainty intervals of experimental results, we review the issue of uncertainty propagation. We show that, using a probabilistic approach (the so-called Bayesian…
We consider the problem of performing Bayesian inference in probabilistic models where observations are accompanied by uncertainty, referred to as "uncertain evidence." We explore how to interpret uncertain evidence, and by extension the…
I recall my 'matter-gravity entanglement hypothesis' and briefly review the evidence for it, based partly on its seeming ability to resolve a number of puzzles related to quantum black holes including the black hole information loss puzzle.…
Machine learning with application to questions in the physical sciences has become a widely used tool, successfully applied to classification, regression and optimization tasks in many areas. Research focus mostly lies in improving the…
Maximum entropy principle identifies forces conjugated to observables and the thermodynamic relations between them, independent upon their underlying mechanistic details. For data about state distributions or transition statistics, the…
We study the informational underpinnings of thermodynamics and statistical mechanics, using an abstract framework, general probabilistic theories, capable of describing arbitrary physical theories. This allows one to abstract the…
In this paper, we analyze the relationship between entropy and information in the context of the mixing process of two identical ideal gases. We will argue that entropy has a special information-based feature that is enfolded in the…
This short book is an elementary course on entropy, leading up to a calculation of the entropy of hydrogen gas at standard temperature and pressure. Topics covered include information, Shannon entropy and Gibbs entropy, the principle of…
Several different uncertain inference systems (UISs) have been developed for representing uncertainty in rule-based expert systems. Some of these, such as Mycin's Certainty Factors, Prospector, and Bayes' Networks were designed as…
Any problem is concerned with the mind, but what do minds make a decision on? Here we show that there are three conditions for the mind to make a certain answer. We found that some difficulties in physics and mathematics are in fact…
We take another look at the general problem of selecting a preferred probability measure among those that comply with some given constraints. The dominant role that entropy maximization has obtained in this context is questioned by arguing…