Related papers: Information functionals and the notion of (un)cert…
A class of distortions termed functional Bregman divergences is defined, which includes squared error and relative entropy. A functional Bregman divergence acts on functions or distributions, and generalizes the standard Bregman divergence…
The maximum entropy principle can be used to assign utility values when only partial information is available about the decision maker's preferences. In order to obtain such utility values it is necessary to establish an analogy between…
Information theory provides principled ways to analyze different inference and learning problems such as hypothesis testing, clustering, dimensionality reduction, classification, among others. However, the use of information theoretic…
Determining functionals are tools to describe the finite dimensional long-term dynamics of infinite dimensional dynamical systems. There also exist several applications to infinite dimensional {\em random} dynamical systems. In these…
Shannon based his information theory on the notion of probability measures as it we developed by Kolmogorov. In this paper we study some fundamental problems in information theory based on expectation measures. In the theory of expectation…
We propose R\'enyi information generating function and discuss its properties. A connection between the R\'enyi information generating function and the diversity index is proposed for discrete type random variables. The relation between the…
The work is devoted to study of the following problem: can we use any qualitative criteria for realization of such universal phenomenon as self-organization in open systems? We have defined values of information at fixed points of…
Partial orders have been used to model several experimental setups, going from classical thermodynamics and general relativity to the quantum realm with its resource theories. In order to study such experimental setups, one typically…
We proposed a Least Information theory (LIT) to quantify meaning of information in probability distribution changes, from which a new information retrieval model was developed. We observed several important characteristics of the proposed…
The random matrix ensembles (RME) of quantum statistical Hamiltonians, e.g. Gaussian random matrix ensembles (GRME) and Ginibre random matrix ensembles (Ginibre RME), are applied in literature to following quantum statistical systems:…
A general problem of optimal information acquisition for its use in decision making problems is considered. This motivates the need for developing quantitative measures of information sources' capabilities for supplying accurate information…
Markov categories are a novel framework to describe and treat problems in probability and information theory. In this work we combine the categorical formalism with the traditional quantitative notions of entropy, mutual information, and…
This work proposes a new loss function targeting classification problems, utilizing a source of information overlooked by cross entropy loss. First, we derive a series of the tightest upper and lower bounds for the probability of a random…
We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…
In this paper, we develop a framework for information theoretic learning based on infinitely divisible matrices. We formulate an entropy-like functional on positive definite matrices based on Renyi's axiomatic definition of entropy and…
In spite of recent contributions to the literature, informative cluster size settings are not well known and understood. In this paper, we give a formal definition of the problem and describe it from different viewpoints. Data generating…
Several tasks in information retrieval (IR) rely on assumptions regarding the distribution of some property (such as term frequency) in the data being processed. This thesis argues that such distributional assumptions can lead to incorrect…
We carry out a systematic study of uncertainty measures that are generic to dynamical processes of varied origins, provided they induce suitable continuous probability distributions. The major technical tool are the information theory…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
It is well known that a Shannon based definition of information entropy leads in the classical case to the Boltzmann entropy. It is tempting to regard the Von Neumann entropy as the corresponding quantum mechanical definition. But the…