Related papers: On the Uniform Distribution of Regular Expressions
A large set of signals can sometimes be described sparsely using a dictionary, that is, every element can be represented as a linear combination of few elements from the dictionary. Algorithms for various signal processing applications,…
We consider covariance parameter estimation for Gaussian processes with functional inputs. From an increasing-domain asymptotics perspective, we prove the asymptotic consistency and normality of the maximum likelihood estimator. We extend…
We revisit the replica method for analyzing inference and learning in parametric models, considering situations where the data-generating distribution is unknown or analytically intractable. Instead of assuming idealized distributions to…
In the synthesis model signals are represented as a sparse combinations of atoms from a dictionary. Dictionary learning describes the acquisition process of the underlying dictionary for a given set of training samples. While ideally this…
The diffusion probabilistic generative models are widely used to generate high-quality data. Though they can synthetic data that does not exist in the training set, the rationale behind such generalization is still unexplored. In this…
We present a general framework for studying regularized estimators; such estimators are pervasive in estimation problems wherein "plug-in" type estimators are either ill-defined or ill-behaved. Within this framework, we derive, under…
Large, human-annotated datasets are central to the development of natural language processing models. Collecting these datasets can be the most challenging part of the development process. We address this problem by introducing a general…
Several methods are discussed that construct a finite automaton given a context-free grammar, including both methods that lead to subsets and those that lead to supersets of the original context-free language. Some of these methods of…
Consider the situation where a word is chosen probabilistically from a finite list. If an attacker knows the list and can inquire about each word in turn, then selecting the word via the uniform distribution maximizes the attacker's…
Diffusion probabilistic models have become a cornerstone of modern generative AI, yet the mechanisms underlying their generalization remain poorly understood. In fact, if these models were perfectly minimizing their training loss, they…
Generalized Zeckendorf decompositions are expansions of integers as sums of elements of solutions to recurrence relations. The simplest cases are base-$b$ expansions, and the standard Zeckendorf decomposition uses the Fibonacci sequence.…
Most expressivity results for transformers treat them as language recognizers -- devices that accept or reject strings -- rather than as they are used in practice: as language models that generate strings autoregressively and…
For a variety of pattern-avoiding classes, we describe the limiting distribution for the number of fixed points for involutions chosen uniformly at random from that class. In particular we consider monotone patterns of arbitrary length as…
Probabilistic word embeddings have shown effectiveness in capturing notions of generality and entailment, but there is very little work on doing the analogous type of investigation for sentences. In this paper we define probabilistic models…
In this document we achieve exact and asymptotic enumeration of words, compositions over a finite group, and/or integer compositions characterized by local restrictions and, separately, subsequence pattern avoidance. We also count…
The rate of occurrence of words is not uniform but varies from document to document. Despite this observation, parameters for conventional n-gram language models are usually derived using the assumption of a constant word rate. In this…
We discuss in detail the asymptotic distribution of sample expectiles. First, we show uniform consistency under the assumption of a finite mean. In case of a finite second moment, we show that for expectiles other then the mean, only the…
We discuss an acceptance-rejection algorithm for the random number generation from the Kolmogorov distribution. Since the cumulative distribution function (CDF) is expressed as a series, in order to obtain the density function we need to…
We consider "nonconventional" averaging setup in the form $\frac {dX^\epsilon(t)}{dt}=\epsilon B\big(X^\epsilon(t),\xi(q_1(t)), \xi(q_2(t)),...,\xi(q_\ell(t))\big)$ where $\xi(t),t\geq 0$ is either a stochastic process or a dynamical system…
We consider a random permutation drawn from the set of permutations of length $n$ that avoid some given set of patterns of length 3. We show that the number of occurrences of another pattern $\sigma$ has a limit distribution, after suitable…