Related papers: Maximum Relative Divergence Principle for Grading …
We investigate the theoretical foundations of a recently introduced entropy-based formulation of weighted least squares for the approximation of overdetermined linear systems, motivated by robust data fitting in the presence of sparse gross…
We consider the uncertainty between two pairs of local projective measurements performed on a multipartite system. We show that the optimal bound in any linear uncertainty relation, formulated in terms of the Shannon entropy, is additive.…
Entropy is critically examined as a fundamental concept in contemporary science and informatics. Although the typical Shannon entropy provides a proper framework for describing the canonical ensemble, it fails to represent adequately the…
The paper describes an approach to measuring convergence of an algorithm to its result in terms of an entropy-like function of partitions of its inputs of a given length. The goal is to look at the algorithmic data processing from the…
We provide new perspectives and inference algorithms for Maximum Entropy (MaxEnt) Inverse Reinforcement Learning (IRL), which provides a principled method to find a most non-committal reward function consistent with given expert…
The Maximum Entropy Reinforcement Learning (MaxEnt RL) framework is a leading approach for achieving efficient learning and robust performance across many RL tasks. However, MaxEnt methods have also been shown to struggle with…
The Principle of Maximum Entropy, a powerful and general method for inferring the distribution function given a set of constraints, is applied to deduce the overall distribution of 3D plasmoids (flux ropes/tubes) for systems where resistive…
We consider the problem of defining the significance of an itemset. We say that the itemset is significant if we are surprised by its frequency when compared to the frequencies of its sub-itemsets. In other words, we estimate the frequency…
Maximum entropy modeling is a flexible and popular framework for formulating statistical models given partial knowledge. In this paper, rather than the traditional method of optimizing over the continuous density directly, we learn a smooth…
We discuss how the method of maximum entropy, MaxEnt, can be extended beyond its original scope, as a rule to assign a probability distribution, to a full-fledged method for inductive inference. The main concept is the (relative) entropy…
Non Performing Asset(NPA) has been in a serious attention by banks over the past few years. NPA cause a huge loss to the banks hence it becomes an extremely critical step in deciding which loans have the capabilities to become an NPA and…
We review here {\it Maximum Caliber} (Max Cal), a general variational principle for inferring distributions of paths in dynamical processes and networks. Max Cal is to dynamical trajectories what the principle of {\it Maximum Entropy} (Max…
Correntropy is a local similarity measure defined in kernel space and the maximum correntropy criterion (MCC) has been successfully applied in many areas of signal processing and machine learning in recent years. The kernel function in…
Relative entropy is a measure of distinguishability for quantum states, and plays a central role in quantum information theory. The family of Renyi entropies generalizes to Renyi relative entropies that include as special cases most entropy…
We show how the Shannon entropy function can be used as a basis to set up complexity measures weighting the economic efficiency of countries and the specialization of products beyond bare diversification. This entropy function guarantees…
We study the notion of approximate entropy within the framework of network theory. Approximate entropy is an uncertainty measure originally proposed in the context of dynamical systems and time series. We firstly define a purely structural…
Given a task of predicting $Y$ from $X$, a loss function $L$, and a set of probability distributions $\Gamma$ on $(X,Y)$, what is the optimal decision rule minimizing the worst-case expected loss over $\Gamma$? In this paper, we address…
The standard definition formula for probabilities of independent events is derived as a consequence of the Insufficient Reason Principle expressed as the Maximum Relative Divergence Principle for grading (order-comonotonic) functions on a…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…
The canonical ensemble describes an open system in equilibrium with a heat bath of fixed temperature. The probability distribution of such a system, the Boltzmann distribution, is derived from the uniform probability distribution of the…