Related papers: Modifications of Simon text model
This paper explores unsupervised learning of parsing models along two directions. First, which models are identifiable from infinite data? We use a general technique for numerically checking identifiability based on the rank of a Jacobian…
This is an alternative version of the course notes in chao-dyn/9407003. The previous version is based on measuring the size of lisp s-expressions. This version is based on measuring the size of what I call lisp m-expressions, which are lisp…
While a mature body of work supports the study of rewriting systems, abstract tools for Probabilistic Rewriting are still limited. In this paper we study the question of uniqueness of the result (unique limit distribution), and develop a…
Piecewise testable languages are a subclass of the regular languages. There are many equivalent ways of defining them; Simon's congruence $\sim_k$ is one of the most classical approaches. Two words are $\sim_k$-equivalent if they have the…
Large language models (LLMs) have reached human-like proficiency in generating diverse textual content, underscoring the necessity for effective fake text detection to avoid potential risks such as fake news in social media. Previous…
It is shown that the method of exchangeable pairs introduced by Stein [Approximate Computation of Expectations (1986) IMS, Hayward, CA] for normal approximation can effectively be used for translated Poisson approximation. Introducing an…
We present three large-scale experiments on binary text matching classification task both in Chinese and English to evaluate the effectiveness and generalizability of random text perturbations as a data augmentation approach for NLP. It is…
We present a language independent, unsupervised method for building word embeddings using morphological expansion of text. Our model handles the problem of data sparsity and yields improved word embeddings by relying on training word…
Masked Language Models (MLM) are self-supervised neural networks trained to fill in the blanks in a given sentence with masked tokens. Despite the tremendous success of MLMs for various text based tasks, they are not robust for spoken…
Sentence formation is a highly structured, history-dependent, and sample-space reducing (SSR) process. While the first word in a sentence can be chosen from the entire vocabulary, typically, the freedom of choosing subsequent words gets…
Three extensions and reinterpretations of nonclassical probabilities are reviewed. (i) We propose to generalize the probability axiom of quantum mechanics to self-adjoint positive operators of trace one. Furthermore, we discuss the…
The recent proliferation of research into transformer based natural language processing has led to a number of studies which attempt to detect the presence of human-like cognitive behavior in the models. We contend that, as is true of human…
While word embeddings are currently predominant for natural language processing, most of existing models learn them solely from their contexts. However, these context-based word embeddings are limited since not all words' meaning can be…
We present a new approach to encourage neural machine translation to satisfy lexical constraints. Our method acts at the training step and thereby avoiding the introduction of any extra computational overhead at inference step. The proposed…
Simple representations of documents based on the occurrences of terms are ubiquitous in areas like Information Retrieval, and also frequent in Natural Language Processing. In this work we propose a logical-probabilistic approach to the…
Word order evolution has been hypothesized to be constrained by a word order permutation ring: transitions involving orders that are closer in the permutation ring are more likely. The hypothesis can be seen as a particular case of…
We consider the satisfiability problem for the two-variable fragment of the first-order logic extended with modulo counting quantifiers and interpreted over finite words or trees. We prove a small-model property of this logic, which gives a…
Permutations are usually enumerated by size, but new results can be found by enumerating them by inversions instead, in which case one must restrict one's attention to indecomposable permutations. In the style of the seminal paper by Simion…
There is a lack of an evaluation methodology that estimates the extent to which large language models (LLMs) use code-switching (CS) in the same way as bilinguals. Existing methods do not have wide language coverage, fail to account for the…
Summation formulae are classical tools in analysis: Taylor-MacLaurin, Euler-MacLaurin, Poisson, Vorono\"i, Circle formulae\ldots We will show how, from a single equation - referred to as the mother-equation - it is possible to unify these…