Related papers: The EL Theorem
We propose an efficient algorithm for approximate computation of the profile maximum likelihood (PML), a variant of maximum likelihood maximizing the probability of observing a sufficient statistic rather than the empirical sample. The PML…
Multilayer (or multiple) networks are widely used to represent diverse patterns of relationships among objects in increasingly complex real-world systems. Identifying a common invariant subspace across network layers has become an active…
Maximum likelihood learning with exponential families leads to moment-matching of the sufficient statistics, a classic result. This can be generalized to conditional exponential families and/or when there are hidden data. This document…
A very simple but useful almost sure convergence theorem of probability is given.
An $n$-tuple $D=(d(1),\dots,d(n))$ is a \emph{feasible degree sequence} if there is a graph on $\{1,\dots,n\}$ such that $i$ has degree $d(i)$. Any such graph will have $m=\sum_{i=1}^n d(i)/2$ edges. Letting $G(D)$ be a graph chosen…
We study the small ball probability of an order-$\ell$ simple random tensor $X=X^{(1)}\otimes\cdots\otimes X^{(\ell)}$ where $X^{(i)}, 1\leq i\leq\ell$ are independent random vectors in $\mathbb{R}^n$ that are log-concave or have…
Maximum likelihood estimation is a fundamental computational problem in statistics. In this note, we give a bound for the maximum likelihood degree of algebraic statistical models for discrete data. As usual, such models are identified with…
For a sequence $\{X_{n}, \, n \geqslant 1 \}$ of random variables satisfying $\mathbb{E} \lvert X_{n} \rvert < \infty$ for all $n \geqslant 1$, a maximal inequality is established, and used to obtain strong law of large numbers for…
A well-known conjecture in analytic number theory states that for every pair of sets $X,Y\subset\mathbb{Z}/p\mathbb{Z}$, each of size at least $\log ^C p$ (for some constant $C$) we have that the number of pairs $(x,y)\in X\times Y$ such…
We prove an analogue of the classical ballot theorem that holds for any random walk in the range of attraction of the normal distribution. Our result is best possible: we exhibit examples demonstrating that if any of our hypotheses are…
We generalise the Erdos-Renyi limit theorem on the maximum of the partial sums of random variables to the case when the number of terms in these sums is randomly distributed. Certain relations between the limiting theorems of this type and…
For general data, the number of complex solutions to the likelihood equations is constant and this number is called the (maximum likelihood) ML-degree of the model. In this article, we describe the special locus of data for which the…
The inducibility of a graph $H$ measures the maximum number of induced copies of $H$ a large graph $G$ can have. Generalizing this notion, we study how many induced subgraphs of fixed order $k$ and size $\ell$ a large graph $G$ on $n$…
The causal boundary of string propagation -- defined as the hypersurface in loop space bordering the timelike(spacelike) domains in which two successive measurements of the string field do(do not) interfere with one another -- is argued to…
Given a random process $x(\tau)$ which undergoes stochastic resetting at a constant rate $r$ to a position drawn from a distribution ${\cal P}(x)$, we consider a sequence of dynamical observables $A_1, \dots, A_n$ associated to the…
Maximum regularized likelihood estimators (MRLEs) are arguably the most established class of estimators in high-dimensional statistics. In this paper, we derive guarantees for MRLEs in Kullback-Leibler divergence, a general measure of…
We investigate the probability for the largest segment in with total displacement $Q$ in an $N$-step random walk to have length $L$. Using analytical, exact enumeration, and Monte Carlo methods, we reveal the complex structure of the…
The expectation-maximization (EM) algorithm and its variants are widely used in statistics. In high-dimensional mixture linear regression, the model is assumed to be a finite mixture of linear regression and the number of predictors is much…
Let $L$ be a set of $n$ lines in $\reals^d$, for $d\ge 3$. A {\em joint} of $L$ is a point incident to at least $d$ lines of $L$, not all in a common hyperplane. Using a very simple algebraic proof technique, we show that the maximum…
We develop a general framework for proving rigorous guarantees on the performance of the EM algorithm and a variant known as gradient EM. Our analysis is divided into two parts: a treatment of these algorithms at the population level (in…