Related papers: The EL Theorem
Expectation Maximization (EM) is among the most popular algorithms for estimating parameters of statistical models. However, EM, which is an iterative algorithm based on the maximum likelihood principle, is generally only guaranteed to find…
Given two sequences over a finite alphabet $\mathcal{L}$, the $D_2$ statistic is the number of $m$-letter word matches between the two sequences. This statistic is used in bioinformatics for expressed sequence tag database searches. Here we…
It has been shown that there is a sequential embedding structure in a $w_N$\ string theory based on a linearized $W_N$\ algebra. The $w_N$\ string theory is obtained as a special realization of the $w_{N+1}$\ string. The $w_{\infty}$\…
We present novel bounds for estimating discrete probability distributions under the $\ell_\infty$ norm. These are nearly optimal in various precise senses, including a kind of instance-optimality. Our data-dependent convergence guarantees…
Frankl's union-closed sets conjecture states that in every finite union-closed set of sets, there is an element that is contained in at least half of the member-sets (provided there are at least two members). The conjecture has an…
The maximum likelihood threshold of a graph is the smallest number of data points that guarantees that maximum likelihood estimates exist almost surely in the Gaussian graphical model associated to the graph. We show that this graph…
Let L contain only the equality symbol and let L^+ be an arbitrary finite symmetric relational language containing L . Suppose probabilities are defined on finite L^+ structures with ''edge probability'' n^{- alpha}. By T^alpha, the almost…
Let $X $ be a square integrable random variable with basic probability space $(\O, \A, \P)$, taking values in a lattice $\mathcal L(v_0,1)=\big\{v_k=v_0+ k,k\in \Z\big\}$ and such that $\t_X =\sum_{k\in \Z}\P\{X=v_k\}\wedge…
This article studies exponential families $\mathcal{E}$ on finite sets such that the information divergence $D(P\|\mathcal{E})$ of an arbitrary probability distribution from $\mathcal{E}$ is bounded by some constant $D>0$. A particular…
We study the set ${\cal L}(G)$ of lengths of all cycles that appear in a random $d$-regular $G$ on $n$ vertices for a fixed $d\geq 3$, as well as in Erd\H{o}s--R\'enyi random graphs on $n$ vertices with a fixed average degree $c>1$.…
The probability distributions for charged particle numbers and their densities are derived in statistical ensembles with conservation laws. It is shown that if this limit is properly taken then the canonical and grand canonical ensembles…
In this note we expose some surprising connections between string theory and statistical inference. We consider a large collective of agents sweeping out a family of nearby statistical models for an M-dimensional manifold of statistical…
We examine the {combinatorial} or {probabilistic} definition ("Boltzmann's principle") of the entropy or cross-entropy function $H \propto \ln \mathbb{W}$ or $D \propto - \ln \mathbb{P}$, where $\mathbb{W}$ is the statistical weight and…
The probability distribution P from which the history of our universe is sampled represents a theory of everything or TOE. We assume P is formally describable. Since most (uncountably many) distributions are not, this imposes a strong…
The advent of data science has spurred interest in estimating properties of distributions over large alphabets. Fundamental symmetric properties such as support size, support coverage, entropy, and proximity to uniformity, received most…
We give upper bounds for the number $\Phi_\ell(G)$ of matchings of size $\ell$ in (i) bipartite graphs $G=(X\cup Y, E)$ with specified degrees $d_x$ ($x\in X$), and (ii) general graphs $G=(V,E)$ with all degrees specified. In particular,…
Recent work on percolation in $d=2$ [J. Phys. A {\bf 55} 204002] introduced an operator that gives a weight $k^{\ell}$ to configurations with $\ell$ `nested paths' (NP), i.e. disjoint cycles surrounding the origin, if there exists a cluster…
We give answer to an open problem regarding consistency of the maximum likelihood estimators (MLEs) in generalized linear mixed models (GLMMs) involving crossed random effects. The solution to the open problem introduces an interesting,…
Maximum-likelihood estimation (MLE) is arguably the most important tool for statisticians, and many methods have been developed to find the MLE. We present a new inequality involving posterior distributions of a latent variable that holds…
Let $D$ be a dictionary in a Hilbert space $H$, that is, a set of unit elements whose linear combinations are dense in $H$. We consider the least $m$-term deviation $\sigma_m(x)$ of an element $x\in H$: this is the distance of $x$ from the…