Related papers: On the Uniform Distribution of Regular Expressions
While state-of-the-art neural network models continue to achieve lower perplexity scores on language modeling benchmarks, it remains unknown whether optimizing for broad-coverage predictive performance leads to human-like syntactic…
Large language models (LLMs) have recently gained much popularity due to their surprising ability at generating human-like English sentences. LLMs are essentially predictors, estimating the probability of a sequence of words given the past.…
We extend balloon and sample-smoothing estimators, two types of variable-bandwidth kernel density estimators, by a shift parameter and derive their asymptotic properties. Our approach facilitates the unified study of a wide range of density…
Normal forms for wide classes of closed IL formulas were given in [4]. Here we quantify asymptotically, in exact numbers, how wide those classes are. As a consequence, we show that the "majority" of closed IL formulas have GL-equivalents,…
The extension of pattern avoidance from ordinary permutations to those on multisets gave birth to several interesting enumerative results. We study permutations on regular multisets, i.e., multisets in which each element occurs the same…
A telegraph process with an elastic barrier at the origin was studied in [5]; in particular the number of visits of the origin before the absorption is a geometric distributed random variable M. Some asymptotic results (large and moderate…
We present a setup for training, evaluating and interpreting neural language models, that uses artificial, language-like data. The data is generated using a massive probabilistic grammar (based on state-split PCFGs), that is itself derived…
The minimal deterministic finite automaton is generally used to determine regular languages equality. Antimirov and Mosses proposed a rewrite system for deciding regular expressions equivalence of which Almeida et al. presented an improved…
This paper is devoted to the estimators of the mean that provide strong non-asymptotic guarantees under minimal assumptions on the underlying distribution. The main ideas behind proposed techniques are based on bridging the notions of…
This paper proposes a new approach to duration modelling for statistical parametric speech synthesis in which a recurrent statistical model is trained to output a phone transition probability at each timestep (acoustic frame). Unlike…
We study the asymptotic properties of distributed consensus algorithms over switching directed random networks. More specifically, we focus on consensus algorithms over independent and identically distributed, directed random graphs, where…
There is an ongoing debate on whether neural networks can grasp the quasi-regularities in languages like humans. In a typical quasi-regularity task, English past tense inflections, the neural network model has long been criticized that it…
We study the variance of the number of zeroes of a stationary Gaussian process on a long interval. We give a simple asymptotic description under mild mixing conditions. This allows us to characterise minimal and maximal growth. We show that…
When formulated as an unsupervised learning problem, anomaly detection often requires a model to learn the distribution of normal data. Previous works apply Generative Adversarial Networks (GANs) to anomaly detection tasks and show good…
In this paper, we present the asymptotic distribution of M-estimators for parameters in non-stationary AR(p) processes. The innovations are assumed to be in the domain of attraction of a stable law with index $0<\alpha\le2$. In particular,…
The method to derive uniform bounds with Gaussian and Rademacher complexities is extended to the case where the sample average is replaced by a nonlinear statistic. Tight bounds are obtained for U-statistics, smoothened L-statistics and…
This article studies the asymptotic properties of Bayesian or frequentist estimators of a vector of parameters related to structural properties of sequences of graphs. The estimators studied originate from a particular class of graphex…
At its core, machine learning seeks to train models that reliably generalize beyond noisy observations; however, the theoretical vacuum in which state-of-the-art universal approximation theorems (UATs) operate isolates them from this goal,…
In $M$-estimation under standard asymptotics, the weak convergence combined with the polynomial type large deviation estimate of the associated statistical random field Yoshida (2011) provides us with not only the asymptotic distribution of…
This paper proposes a new notion of typical sequences on a wide class of abstract alphabets (so-called standard Borel spaces), which is based on approximations of memoryless sources by empirical distributions uniformly over a class of…