Related papers: Upper Bound on Normalized Maximum Likelihood Codes…
Mechanistic network models specify the mechanisms by which networks grow and change, allowing researchers to investigate complex systems using both simulation and analytical techniques. Unfortunately, it is difficult to write likelihoods…
Monte Carlo maximum likelihood (MCML) provides an elegant approach to find maximum likelihood estimators (MLEs) for latent variable models. However, MCML algorithms are computationally expensive when the latent variables are…
Mixtures-of-Experts models and their maximum likelihood estimation (MLE) via the EM algorithm have been thoroughly studied in the statistics and machine learning literature. They are subject of a growing investigation in the context of…
In this work, we prove new results concerning the combinatorial properties of random linear codes. Firstly, we prove a lower bound on the list-size required for random linear codes over $\mathbb F_q$ $\varepsilon$-close to capacity to…
The variable-length source coding problem allowing the error probability up to some constant is considered for general sources. In this problem the optimum mean codeword length of variable-length codes has already been determined. On the…
Mixtures of generalized normal distributions (MGND) have gained popularity for modelling datasets with complex statistical behaviours. However, the estimation of the shape parameter within the maximum likelihood framework is quite complex,…
The Predictive Normalized Maximum Likelihood (pNML) scheme has been recently suggested for universal learning in the individual setting, where both the training and test samples are individual data. The goal of universal learning is to…
We present a generalization of the maximal inequalities that upper bound the expectation of the maximum of $n$ jointly distributed random variables. We control the expectation of a randomly selected random variable from $n$ jointly…
This note aims to demonstrate that performing maximum-likelihood estimation for a mixture model is equivalent to minimizing over the parameters an optimal transport problem with entropic regularization. The objective is pedagogical: we seek…
We construct an optimal quantum universal variable-length code that achieves the admissible minimum rate, i.e., our code is used for any probability distribution of quantum states. Its probability of exceeding the admissible minimum rate…
We establish a general formula for the maximum size of finite length block codes with minimum pairwise distance no less than $d$. The achievability argument involves an iterative construction of a set of radius-$d$ balls, each centered at a…
In supervised batch learning, the predictive normalized maximum likelihood (pNML) has been proposed as the min-max regret solution for the distribution-free setting, where no distributional assumptions are made on the data. However, the…
We consider distributed estimation of the inverse covariance matrix, also called the concentration or precision matrix, in Gaussian graphical models. Traditional centralized estimation often requires global inference of the covariance…
We describe a general technique that yields the first {\em Statistical Query lower bounds} for a range of fundamental high-dimensional learning problems involving Gaussian distributions. Our main results are for the problems of (1) learning…
We consider the maximum coding rate achievable by uniformly-random codes for the deletion channel. We prove an upper bound that's within 0.1 of the best known lower bounds for all values of the deletion probability $d,$ and much closer for…
We prove upper bounds on the rate, called "mixing rate", at which the von Neumann entropy of the expected density operator of a given ensemble of states changes under non-local unitary evolution. For an ensemble consisting of two states,…
This paper is concerned with bounds on the maximum-likelihood (ML) decoding error probability of Reed-Solomon (RS) codes over additive white Gaussian noise (AWGN) channels. To resolve the difficulty caused by the dependence of the Euclidean…
This paper considers the average complexity of maximum likelihood (ML) decoding of convolutional codes. ML decoding can be modeled as finding the most probable path taken through a Markov graph. Integrated with the Viterbi algorithm (VA),…
In this paper, we considier the limiting distribution of the maximum interpoint Euclidean distance $M_n=\max _{1 \leq i<j \leq n}\left\|\boldsymbol{X}_i-\boldsymbol{X}_j\right\|$, where $\boldsymbol{X}_1, \boldsymbol{X}_2, \ldots,…
Clustering mixtures of Gaussian distributions is a fundamental and challenging problem that is ubiquitous in various high-dimensional data processing tasks. While state-of-the-art work on learning Gaussian mixture models has focused…