Related papers: Mismatched Guesswork
A major hurdle on the road to conversational interfaces is the difficulty in collecting data that maps language utterances to logical forms. One prominent approach for data collection has been to automatically generate pseudo-language…
We introduce probabilistic language tries (PLTs), a unified representation that makes explicit the prefix structure implicitly defined by any generative model over sequences. By assigning to each outgoing edge the conditional probability of…
In prefix coding over an infinite alphabet, methods that consider specific distributions generally consider those that decline more quickly than a power law (e.g., Golomb coding). Particular power-law distributions, however, model many…
We study the asymptotic theory of misspecified models for diffusion processes with noisy nonsynchronous observations. Unlike with correctly specified models, the original maximum-likelihood-type estimator has an asymptotic bias under the…
Arithmetic codes are usually deemed as the most important means to implement lossless source coding, whose principle is mapping every source symbol to a sub-interval in [0, 1). For every source symbol, the length of its mapping sub-interval…
The guessing number of a directed graph (digraph), equivalent to the entropy of that digraph, was introduced as a direct criterion on the solvability of a network coding instance. This paper makes two contributions on the guessing number.…
Typical random codes (TRC) in a communication scenario of source coding with side information at the decoder is the main subject of this work. We study the semi-deterministic code ensemble, which is a certain variant of the ordinary random…
A major bottleneck of standard auto-regressive large language models is that their inference process is inherently sequential, resulting in very long and costly inference times. To circumvent this, practitioners proposed a class of language…
We introduce a universal quantization scheme based on random coding, and we analyze its performance. This scheme consists of a source-independent random codebook (typically_mismatched_ to the source distribution), followed by optimal…
We propose a new method for the calculation of the statistical properties, as e.g. the entropy, of unknown generators of symbolic sequences. The probability distribution p(k) of the elements k of a population can be approximated by the…
The main goal in distributed symmetry-breaking is to understand the locality of problems; i.e., the radius of the neighborhood that a node needs to explore in order to arrive at its part of a global solution. In this work, we study the…
In many applications of natural language processing (NLP) it is necessary to determine the likelihood of a given word combination. For example, a speech recognizer may need to determine which of the two word combinations ``eat a peach'' and…
We consider the problem of learning co-occurrence information between two word categories, or more in general between two discrete random variables taking values in a hierarchically classified domain. In particular, we consider the problem…
Large language models (LLMs) demand considerable computational, energy, and financial resources during both training and deployment. While scaling laws for training have guided much of the field's recent progress, inference costs now…
This paper considers the problem of guessing the realization of a finite alphabet source when some side information is provided. The only knowledge the guesser has about the source and the correlated side information is that the joint…
Some effort has been undertaken over the last decade to provide conditions for the control of the false discovery rate by the linear step-up procedure (LSU) for testing $n$ hypotheses when test statistics are dependent. In this paper we…
Simpson's paradox, a long-standing statistical phenomenon, describes the reversal of an observed association when data are disaggregated into sub-populations. It has critical implications across statistics, epidemiology, economics, and…
The security of code-based cryptography relies primarily on the hardness of generic decoding with linear codes. The best generic decoding algorithms are all improvements of an old algorithm due to Prange: they are known under the name of…
The main idea of nested sampling is to substitute the high-dimensional likelihood integral over the parameter space $\Omega$ by an integral over the unit line $[0,1]$ by employing a push-forward with respect to a suitable transformation.…
We present explicit formulae for parameterized families of probabilities of the number of nonoverlapping words and increasing nonoverlapping words in independent and identically distributed (i.i.d.) finite valued random variables,…