Related papers: Asymptotic Log-loss of Prequential Maximum Likelih…

Given a sequence of $n$ identically distributed random variables with common distribution $F$, the \emph{fragility distribution of order $m$}, represented by $\FD$, is the limit conditional distribution of the number of exceedances given…

Deep neural networks (DNNs) defy the classical bias-variance trade-off: adding parameters to a DNN that interpolates its training data will typically improve its generalization performance. Explaining the mechanism behind this ``benign…

We study parametric inference on a rich class of hazard regression models in the presence of right-censoring. Previous literature has reported some inferential challenges, such as multimodal or flat likelihood surfaces, in this class of…

We consider a variation of the Wyner-Ziv problem pertaining to lossy compression of individual sequences using finite-state encoders and decoders. There are two main results in this paper. The first characterizes the relationship between…

This paper investigates the problem of variable-length lossy source coding allowing a positive excess distortion probability and an overflow probability of codeword lengths. Novel one-shot achievability and converse bounds of the optimal…

Let $M_{l,n}$ be the number of blocks with frequency $l$ in the exchangeable random partition induced by a sample of size $n$ from the Ewens-Pitman sampling model. We show that, as $n$ tends to infinity, $n^{-1}M_{l,n}$ satisfies a large…

Discrete diffusion-based multimodal large language models (dMLLMs) have emerged as a promising alternative to autoregressive MLLMs thanks to their advantages in parallel decoding and bidirectional context modeling, but most existing dMLLMs…

In this paper, an expression for the asymptotic growth rate of the number of small linear-weight codewords of irregular doubly-generalized LDPC (D-GLDPC) codes is derived. The expression is compact and generalizes existing results for LDPC…

Lifted Reed Solomon Codes (Guo, Kopparty, Sudan 2013) were introduced in the context of locally correctable and testable codes. They are multivariate polynomials whose restriction to any line is a codeword of a Reed-Solomon code. We…

For a linear block code C, its stopping redundancy is defined as the smallest number of check nodes in a Tanner graph for C, such that there exist no stopping sets of size smaller than the minimum distance of C. Schwartz and Vardy…

The parametric complexity is the key quantity in the minimum description length (MDL) approach to statistical model selection. Rissanen and others have shown that the parametric complexity of a statistical model approaches a simple function…

Fix a modulus $q$. One would expect the number of primes in each invertible residue class mod $q$ to be multinomially distributed, i.e. for each $p \,\mathrm{mod}\, q$ to behave like an independent random variable uniform on…

Multi-modal Large Langue Models (MLLMs) often process thousands of visual tokens, which consume a significant portion of the context window and impose a substantial computational burden. Prior work has empirically explored visual token…

We investigate certain large deviation asymptotics concerning random interlacements in Z^d, d bigger or equal to 3. We find the principal exponential rate of decay for the probability that the average value of some suitable non-decreasing…

For any admissible value of the parameters $n$ and $k$ there exist $[n,k]$-Maximum Rank distance ${\mathbb F}_q$-linear codes. Indeed, it can be shown that if field extensions large enough are considered, almost all rank distance codes are…

Permutation codes and multi-permutation codes have been widely considered due to their various applications, especially in flash memory. In this paper, we consider permutation codes and multi-permutation codes against a burst of stable…

This paper considers '$\delta$-almost Reed-Muller codes', i.e., linear codes spanned by evaluations of all but a $\delta$ fraction of monomials of degree at most $d$. It is shown that for any $\delta > 0$ and any $\varepsilon>0$, there…

We consider on-line density estimation with a parameterized density from the exponential family. The on-line algorithm receives one example at a time and maintains a parameter that is essentially an average of the past examples. After…

We consider the problem of distilling uniform random bits from an unknown source with a given $p$-entropy using linear hashing. As our main result, we estimate the expected $p$-divergence from the uniform distribution over the ensemble of…

Consider the nonparametric logistic regression problem. In the logistic regression, we usually consider the maximum likelihood estimator, and the excess risk is the expectation of the Kullback-Leibler (KL) divergence between the true and…