Related papers: Why Maximum Entropy? A Non-axiomatic Approach
We introduce a new generalization of relative entropy to non-negative vectors with sums $\gt 1$. We show in a purely combinatorial setting, with no probabilistic considerations, that in the presence of linear constraints defining a convex…
Most of the existing classification methods are aimed at minimization of empirical risk (through some simple point-based error measured with loss function) with added regularization. We propose to approach this problem in a more information…
This paper discusses the application of L1-regularized maximum entropy modeling or SL1-Max [9] to multiclass categorization problems. A new modification to the SL1-Max fast sequential learning algorithm is proposed to handle conditional…
Recently, the conditional maximum-entropy method (abbreviated as C-MaxEnt) has been proposed for selecting priors in Bayesian statistics in a very simple way. Here, it is examined for extreme-value statistics. For the Weibull type as an…
The fundamentals of the Maximum Entropy principle as a rule for assigning and updating probabilities are revisited. The Shannon-Jaynes relative entropy is vindicated as the optimal criterion for use with an updating rule. A constructive…
We study entanglement entropy (EE) for a Maxwell field in 2+1 dimensions. We do numerical calculations in two dimensional lattices. This gives a concrete example of the general results of our recent work on entropy for lattice gauge fields…
We present a differential geometric viewpoint of the quantum MaxEnt estimate of a density operator when only incomplete knowledge encoded in the expectation values of a set of quantum observables is available. Finally, the additional…
Reconstructing 4D or 6D phase space distributions from 1D or 2D measurements is a challenging inverse problem encountered in particle accelerators. Entropy maximization is an established method to incorporate prior information in the…
We consider fitting a bivariate spline regression model to data using a weighted least-squares cost function, with weights that sum to one to form a discrete probability distribution. By applying the principle of maximum entropy, the weight…
For the purpose of causal inference we employ a stochastic model of the data generating process, utilizing individual propensity probabilities for the treatment, and also individual and counterfactual prognosis probabilities for the…
Deep learning achieves remarkable generalization capability with overwhelming number of model parameters. Theoretical understanding of deep learning generalization receives recent attention yet remains not fully explored. This paper…
Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability…
The fields of quantum non-locality in physics, and causal discovery in machine learning, both face the problem of deciding whether observed data is compatible with a presumed causal relationship between the variables (for example a local…
We introduce a new measure of interdependence among the components of a random vector along the main diagonal of the vector copula, i.e. along the line $u_{1}=\ldots=u_{J}$, for $\left(u_{1},\ldots,u_{J}\right)\in\left[0,1\right]^{J}$. Our…
It is often desirable to summarise a probability measure on a space $X$ in terms of a mode, or MAP estimator, i.e.\ a point of maximum probability. Such points can be rigorously defined using masses of metric balls in the small-radius…
The Expectation-Maximization (EM) algorithm for mixture models often results in slow or invalid convergence. The popular convergence proof affirms that the likelihood increases with Q; Q is increasing in the M -step and non-decreasing in…
We characterize the extreme points of the set of incentive-compatible mechanisms for screening problems with linear utility. Our framework subsumes problems with and without transfers, such as monopoly pricing, principal-optimal bilateral…
There exists, in general, a convex set of quantum state estimators that maximize the likelihood for informationally incomplete data. We propose an estimation scheme, catered to measurement data of this kind, to search for the exact…
The standard implementation of the Maximum Entropy Method (MEM) follows Bryan and deploys a Singular Value Decomposition (SVD) to limit the dimensionality of the underlying solution space apriori. Here we present arguments based on the…
Maximum entropy models provide the least constrained probability distributions that reproduce statistical properties of experimental datasets. In this work we characterize the learning dynamics that maximizes the log-likelihood in the case…