Related papers: On Tsallis Entropy Bias and Generalized Maximum En…
State-of-the-art techniques for enhancing robustness of deep networks mostly rely on empirical risk minimization with suitable data augmentation. In this paper, we propose a complementary approach motivated by communication theory, aimed at…
In constrained stochastic optimization, one naturally expects that imposing a stricter feasible set does not increase the statistical risk of an estimator defined by projection onto that set. In this paper, we show that this intuition can…
Method of parameterizing and smoothing the unknown underling distributions using Bernstein polynomials is proposed, verified and investigated. Any distribution with bounded and smooth enough density can be approximated by the proposed…
When an informationally complete measurement is not available, the reconstruction of the density operator that describes the state of a quantum system can be accomplish, in a reliable way, by adopting the maximum entropy principle (MaxEnt…
Predictive inference requires balancing statistical accuracy against informational complexity, yet the choice of complexity measure is usually imposed rather than derived. We treat econometric objects as predictive rules, mappings from…
Compressed Counting (CC)} was recently proposed for approximating the $\alpha$th frequency moments of data streams, for $0<\alpha \leq 2$. Under the relaxed strict-Turnstile model, CC dramatically improves the standard algorithm based on…
We give some results relating asymptotic characterisations of maximum entropy probability measures to characterisations of Bayes optimal classifiers. Our main theorems show that maximum entropy is a universally Bayes optimal decision rule…
We demonstrate that the principle of maximum relative entropy (ME), used judiciously, can ease the specification of priors in model selection problems. The resulting effect is that models that make sharp predictions are disfavoured,…
A general framework with a series of different methods is proposed to improve the estimate of convex function (or functional) values when only noisy observations of the true input are available. Technically, our methods catch the bias…
Entropy is a key measure in studies related to information theory and its many applications. Campbell of the first time recognized that exponential of Shannons entropy is just the size of the sample space when the distribution is uniform.…
The principle of maximum entropy is applied to the spectral analysis of a data signal with general variance matrix and containing gaps in the record. The role of the entropic regularizer is to prevent one from overestimating structure in…
In this tutorial we review the essential arguments behing entropic inference. We focus on the epistemological notion of information and its relation to the Bayesian beliefs of rational agents. The problem of updating from a prior to a…
Recent research has highlighted the practical benefits of subjective interestingness measures, which quantify the novelty or unexpectedness of a pattern when contrasted with any prior information of the data miner (Silberschatz and…
Designing the sensing architecture for large-scale spatio-temporal systems is hard when accuracy requirements are specified but sensor models are uncertain or unavailable. Classical design treats sensor placement and estimation…
Bayesian optimal experimental design provides a principled framework for selecting experimental settings that maximize obtained information. In this work, we focus on estimating the expected information gain in the setting where the…
Preferential sampling is a common feature in geostatistics and occurs when the locations to be sampled are chosen based on information about the phenomena under study. In this case, point pattern models are commonly used as the probability…
It is shown that the distribution derived from the principle of maximum Tsallis entropy is a superposable Levy-type distribution. Concomitantly, the leading order correction to the limit distribution is also deduced. This demonstration…
Entropic tilting (ET) is a Bayesian decision-analytic method for constraining distributions to satisfy defined targets or bounds for sets of expectations. This report recapitulates the foundations and basic theory of ET for conditioning…
Maximum entropy principles in nonextensive statistical physics are revisited as an application of the Tsallis relative entropy defined for non-negative matrices in the framework of matrix analysis. In addtition, some matrix trace…
We revisit the classical problem of inverting dimension-reducing linear mappings using the maximum entropy (MaxEnt) criterion. In the literature, solutions are problem-dependent, inconsistent, and use different entropy measures. We propose…