Related papers: Exponential families from a single KL identity
In a regression setup with deterministic design, we study the pure aggregation problem and introduce a natural extension from the Gaussian distribution to distributions in the exponential family. While this extension bears strong…
Collective graphical models exploit inter-instance associative dependence to output more accurate labelings. However existing models support very limited kind of associativity which restricts accuracy gains. This paper makes two major…
In this paper we show how to exploit interventional data to acquire the joint conditional distribution of all the variables using the Maximum Entropy principle. To this end, we extend the Causal Maximum Entropy method to make use of…
We examine the estimation of the Kullback-Leibler (KL) divergence and the use of the goodness-of-fit test for multivariate continuous distributions. Our starting point is the maximum entropy principle for Shannon entropy: among all…
Bayesian networks (BNs) are a foundational model in machine learning and causal inference. Their graphical structure can handle high-dimensional problems, divide them into a sparse collection of smaller ones, underlies Judea Pearl's…
Modern machine learning embeddings provide powerful compression of high-dimensional data, yet they typically destroy the geometric structure required for classical likelihood-based statistical inference. This paper develops a rigorous…
Recently much attention has been paid to deep generative models, since they have been used to great success for variational inference, generation of complex data types, and more. In most all of these settings, the goal has been to find a…
We present a general method for deriving collapsed variational inference algo- rithms for probabilistic models in the conjugate exponential family. Our method unifies many existing approaches to collapsed variational inference. Our…
We consider partition functions Z(g) = exp (-g(x))dx where g is a nonnegative polynomial action (a degree-2n form) vanishing only at the origin. Such integrals, known as integral discriminants, appear in statistical mechanics, quantum field…
Asymptotics of the normalizing constant is computed for a class of one parameter exponential families on permutations which includes Mallows model with Spearmans's Footrule and Spearman's Rank Correlation Statistic. The MLE, and a…
In [16], under mild conditions, a Wiener-Hopf type factorization is derived for the exponential functional of proper L\'evy processes. In this paper, we extend this factorization by relaxing a finite moment assumption as well as by…
We present explicit classes of probability distributions that can be learned by Restricted Boltzmann Machines (RBMs) depending on the number of units that they contain, and which are representative for the expressive power of the model. We…
The article is devoted to the study of exponential statistical structures of type B, which constitute a subclass of exponential families of probability distributions. This class is characterized by a number of analytical and probabilistic…
We propose extreme value analogues of natural exponential families and exponential dispersion models, and introduce the slope function as an analogue of the variance function. The set of quadratic and power slope functions characterize…
Many complex systems are characterized by non-Boltzmann distribution functions of their statistical variables. If one wants to -- justified or not -- hold on to the maximum entropy principle for complex statistical systems (non-Boltzmann)…
We consider the segmentation problem of univariate distributions from the exponential family with multiple parameters. In segmentation, the choice of the number of segments remains a difficult issue due to the discrete nature of the…
Web image datasets curated online inherently contain ambiguous in-distribution (ID) instances and out-of-distribution (OOD) instances, which we collectively call non-conforming (NC) instances. In many recent approaches for mitigating the…
Fitting a graphical model to a collection of random variables given sample observations is a challenging task if the observed variables are influenced by latent variables, which can induce significant confounding statistical dependencies…
Recent advances have revealed that the rate of convergence of the expected test error in deep supervised learning decays as a function of the intrinsic dimension and not the dimension $d$ of the input space. Existing literature defines this…
We investigate the construction of exponential families from statistical manifolds, a central problem in information geometry. We prove that every compact statistical manifold admits a singular foliation whose leaves are Hessian manifolds.…