Related papers: Language as a matrix product state
The Born rule assigns a probability to any possible outcome of a quantum measurement, but leaves open the question how these probabilities are to be interpreted and, in particular, how they relate to the outcome observed in an actual…
We show that the matrix (or more generally tensor) product states in a finite translation invariant system can be accurately constructed from the same set of local matrices (or tensors) that are determined from an infinite lattice system in…
Large pretrained language models are critical components of modern NLP pipelines. Yet, they suffer from spurious correlations, poor out-of-domain generalization, and biases. Inspired by recent progress in causal machine learning, in…
In this thesis, we present two approaches to a rigorous mathematical and algorithmic foundation of quantitative and statistical inference in constraint-based natural language processing. The first approach, called quantitative constraint…
Recent approaches to cross-lingual word embedding have generally been based on linear transformations between the sets of embedding vectors in the two languages. In this paper, we propose an approach that instead expresses the two…
The Swadesh approach for determining the temporal separation between two languages relies on the stochastic process of words replacement (when a complete new word emerges to represent a given concept). It is well known that the basic…
The use of monoids in the study of word languages recognized by finite-state automata has been quite fruitful. In this work, we look at the same idea of "recognizability by finite monoids" for other monoids. In particular, we attempt to…
Automatic evaluation for open-ended natural language generation tasks remains a challenge. Existing metrics such as BLEU show a low correlation with human judgment. We propose a novel and powerful learning-based evaluation metric:…
The aim of this paper is to extend probability theory from the classical to the product t-norm fuzzy logic setting. More precisely, we axiomatize a generalized notion of finitely additive probability for product logic formulas, called…
The language of probability is used to define several different types of conditional statements. There are four principal types: subjunctive, material, existential, and feasibility. Two further types of conditionals are defined using the…
Zipf's law is just one out of many universal laws proposed to describe statistical regularities in language. Here we review and critically discuss how these laws can be statistically interpreted, fitted, and tested (falsified). The modern…
We present a probabilistic model that simultaneously learns alignments and distributed representations for bilingual data. By marginalizing over word alignments the model captures a larger semantic context than prior work relying on hard…
Rule-based machine translation is more data efficient than the big data-based machine translation approaches, making it appropriate for languages with low bilingual corpus resources -- i.e., minority languages. However, the rule-based…
We argue that language models (LMs) have strong potential as investigative tools for probing the distinction between possible and impossible natural languages and thus uncovering the inductive biases that support human language learning. We…
Recent advancements in large language models have revolutionized text generation with their remarkable capabilities. These models can produce controlled texts that closely adhere to specific requirements when prompted appropriately.…
The Born probability measure describes the statistics of measurements in which observers self-locate themselves in some region of reality. In $\psi$-ontic quantum theories, reality is directly represented by the wavefunction. We show that…
Chinese input recommendation plays an important role in alleviating human cost in typing Chinese words, especially in the scenario of mobile applications. The fundamental problem is to predict the conditional probability of the next word…
Crosslingual transfer is crucial to contemporary language models' multilingual capabilities, but how it occurs is not well understood. We ask what happens to a monolingual language model when it begins to be trained on a second language.…
We describe an incremental unsupervised procedure to learn words from transcribed continuous speech. The algorithm is based on a conservative and traditional statistical model, and results of empirical tests show that it is competitive with…
We consider the problem of computing the probability of regular languages of infinite trees with respect to the natural coin-flipping measure. We propose an algorithm which computes the probability of languages recognizable by \emph{game…