Related papers: Estimation of entropy measures for categorical var…
These lectures deal with the problem of inductive inference, that is, the problem of reasoning under conditions of incomplete information. Is there a general method for handling uncertainty? Or, at least, are there rules that could in…
Various strategies for active learning have been proposed in the machine learning literature. In uncertainty sampling, which is among the most popular approaches, the active learner sequentially queries the label of those instances for…
In its continuous version, the entropy functional measuring the information content of a given probability density may be plagued by a "measure" problem that results from improper weighting of phase space. This issue is addressed…
We introduce a novel entropy-related function, \textit{non-repeatability}, designed to capture dynamical behaviors in complex systems. Its normalized form, \textit{mutability}, has been previously applied in statistical physics as a…
The use of rapidity gaps is proposed as a measure of the spatial pattern of an event. When the event multiplicity is low, the gaps between neighboring particles carry far more information about an event than multiplicity spikes, which may…
The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance…
In nonlinear dynamics, basins of attraction link a given set of initial conditions to its corresponding final states. This notion appears in a broad range of applications where several outcomes are possible, which is a common situation in…
The investor is interested in the expected return and he is also concerned about the risk and the uncertainty assumed by the investment. One of the most popular concepts used to measure the risk and the uncertainty is the variance and/or…
We introduce a new information-theoretic formulation of quantum measurement uncertainty relations, based on the notion of relative entropy between measurement probabilities. In the case of a finite-dimensional system and for any approximate…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
Entropy arises in strong interactions by a dynamical separation of ``partons'' from unobservable ``environment'' modes due to confinement. For interacting scalar fields we calculate the statistical entropy of the observable subsystem.…
Mixture distributions are a workhorse model for multimodal data in information theory, signal processing, and machine learning. Yet even when each component density is simple, the differential entropy of the mixture is notoriously hard to…
Due to spatial dependence -- often characterized as complex and non-linear -- model misspecification is a prevalent and critical issue in spatial data analysis and prediction. As the data, and thus model performance, is heterogeneous,…
Prediction polling is an increasingly popular form of crowdsourcing in which multiple participants estimate the probability or magnitude of some future event. These estimates are then aggregated into a single forecast. Historically,…
Human-generated categorical annotations frequently produce empirical response distributions (soft labels) that reflect ambiguity rather than simple annotator error. We introduce an ambiguity measure that maps a discrete response…
Modelling a complex system is almost invariably a challenging task. The incorporation of experimental observations can be used to improve the quality of a model, and thus to obtain better predictions about the behavior of the corresponding…
Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order…
The problem addressed concerns the determination of the average number of successive attempts of guessing a word of a certain length consisting of letters with given probabilities of occurrence. Both first- and second-order approximations…
The software Randentropy is designed to estimate inequality in a random system where several individuals interact moving among many communities and producing dependent random quantities of an attribute. The overall inequality is assessed by…
Potentially influential spaces in the spatial networks of cities can be detected by means of the entropy participation ratios. Local (connectivity) and global (centrality) entropies are considered. While the connectivity entropy has a…