Related papers: Maximum Entropy Distributions: Bit Complexity and …
We give a characterization of Maximum Entropy/Minimum Relative Entropy inference by providing two `strong entropy concentration' theorems. These theorems unify and generalize Jaynes' `concentration phenomenon' and Van Campenhout and Cover's…
The region of entropic vectors is a convex cone that has been shown to be at the core of many fundamental limits for problems in multiterminal data compression, network coding, and multimedia transmission. This cone has been shown to be…
A new nonparametric model of maximum-entropy (MaxEnt) copula density function is proposed, which offers the following advantages: (i) it is valid for mixed random vector. By `mixed' we mean the method works for any combination of discrete…
We revisit the maximum-entropy inference of the state of a finite-level quantum system under linear constraints. The constraints are specified by the expected values of a set of fixed observables. We point out the existence of…
Entropically regularized optimal transport between probability measures supported on compact subsets of Euclidean space admits a representation as an information projection under moment inequality constraints. Exploiting this structure, I…
We investigate the maximum-entropy model $\mathcal{B}_{n,m,p}$ for random $n$-vertex, $m$-edge multi-hypergraphs with expected edge size $pn$. We show that the expected size of the minimization of $\mathcal{B}_{n,m,p}$, i.e., the number of…
This investigation is a part of a research program aiming to characterize the extreme behavior possible in hydrodynamic models by analyzing the maximum growth of certain fundamental quantities. We consider here the rate of growth of the…
The method of maximum entropy (ME) is extended to address the following problem: Once one accepts that the ME distribution is to be preferred over all others, the question is to what extent are distributions with lower entropy supposed to…
We study the maximum achievable differential entropy at the output of a system assigning to each input X the sum X+N, with N a given noise with probability law absolutely continuous with respect to the Lebesgue measure and where the input…
Inspired by applications to theories of coding and communication in networks of nervous tissue, we study maximum entropy distributions on weighted graphs with a given expected degree sequence. These distributions are characterized by…
In this work, we consider a recently proposed entropy S (called varentropy) defined by a variational relationship dI=beta*(d<x>-<dx>) as a measure of uncertainty of random variable x. By definition, varentropy underlies a generalized…
Under general multivariate regular variation conditions, the extreme Value-at-Risk of a portfolio can be expressed as an integral of a known kernel with respect to a generally unknown spectral measure supported on the unit simplex. The…
When an expert operates a perilous dynamic system, ideal constraint information is tacitly contained in their demonstrated trajectories and controls. The likelihood of these demonstrations can be computed, given the system dynamics and task…
Parameter estimation in Markov random fields (MRFs) is a difficult task, in which inference over the network is run in the inner loop of a gradient descent procedure. Replacing exact inference with approximate methods such as loopy belief…
The aim of this paper is to provide several novel upper bounds on the excess risk with a primal focus on classification problems. We suggest two approaches and the obtained bounds are represented via the distribution dependent local…
Mixed-integer linear programming (MILP) is at the core of many advanced algorithms for solving fundamental problems in combinatorial optimization. The complexity of solving MILPs directly correlates with their support size, which is the…
The mutual information (MI) between two random variables is an important correlation measure in data analysis. The Shannon entropy of a joint probability distribution is the variable part under fixed marginals. We aim to minimize and…
We study the multivariate nonparametric change point detection problem, where the data are a sequence of independent $p$-dimensional random vectors whose distributions are piecewise-constant with Lipschitz densities changing at unknown…
An explosive percolation transition is the abrupt emergence of a giant cluster at a threshold caused by a suppression of the growth of large clusters. In this paper, we consider the information entropy of the cluster size distribution,…
Minimization of the (regularized) entropy of classification probabilities is a versatile class of discriminative clustering methods. The classification probabilities are usually defined through the use of some classical losses from…