Related papers: Tree morphisms, transducers, and integer sequences
Tree sets are abstract structures that can be used to model various tree-shaped objects in combinatorics. Finite tree sets can be represented by finite graph-theoretical trees. We extend this representation theory to infinite tree sets.…
Transformer networks are the de facto standard architecture in natural language processing. To date, there are no theoretical analyses of the Transformer's ability to capture tree structures. We focus on the ability of Transformer networks…
We consider the synthesis of deterministic tree transducers from automaton definable specifications, given as binary relations, over finite trees. We consider the case of specifications that are deterministic top-down tree automatic,…
Morphisms are homomorphisms under the concatenation operation of the set of words over a finite set. Changing the elements of the finite set does not essentially change the morphism. We propose a way to select a unique representing member…
We explore a physical model of ordered sums of integers as trains of rods. The trains for a fixed, possibly infinite, set of rod lengths naturally correspond to nodes in a tree; relations among finite linear recursions encoded in the…
We establish connections between the Transformer architecture, originally introduced for natural language processing, and Graph Neural Networks (GNNs) for representation learning on graphs. We show how Transformers can be viewed as message…
Nobody knows how language works, but many theories abound. Transformers are a class of neural networks that process language automatically with more success than alternatives, both those based on neural computations and those that rely on…
An automatic sequence is a letter-to-letter coding of a fixed point of a uniform morphism. More generally, we have morphic sequences, which are letter-to-letter codings of fixed points of arbitrary morphisms. There are many examples where…
The meaning of a sentence is a function of the relations that hold between its words. We instantiate this relational view of semantics in a series of neural models based on variants of relation networks (RNs) which represent a set of…
Machine transliteration is the process of automatically transforming the script of a word from a source language to a target language, while preserving pronunciation. Sequence to sequence learning has recently emerged as a new paradigm in…
Human language is known to exhibit a nested, hierarchical structure, allowing us to form complex sentences out of smaller pieces. However, many state-of-the-art neural networks models such as Transformers have no explicit hierarchical…
The decidability of equivalence for three important classes of tree transducers is discussed. Each class can be obtained as a natural restriction of deterministic macro tree transducers (MTTs): (1) no context parameters, i.e., top-down tree…
It is well known that attributed tree transducers can be equipped with "regular look-around" in order to obtain a more robust class of translations. We present two characterizations of this class in terms of macro tree transducers (MTTs):…
Many machine learning tasks can be expressed as the transformation---or \emph{transduction}---of input sequences into output sequences: speech recognition, machine translation, protein secondary structure prediction and text-to-speech to…
Regular sequences are natural generalisations of fixed points of constant-length substitutions on finite alphabets, that is, of automatic sequences. Using the harmonic analysis of measures associated with substitutions as motivation, we…
Several abstract machines that operate on symbolic input alphabets have been proposed in the last decade, for example, symbolic automata or lattice automata. Applications of these types of automata include software security analysis and…
The precise formulation of derivation for tree-adjoining grammars has important ramifications for a wide variety of uses of the formalism, from syntactic analysis to semantic interpretation and statistical language modeling. We argue that…
We study the class of quasi-alphabetic relations, i.e., tree transformations defined by tree bimorphisms with two quasi-alphabetic tree homomorphisms and a regular tree language. We present a canonical representation of these relations; as…
Many character-level tasks can be framed as sequence-to-sequence transduction, where the target is a word from a natural language. We show that leveraging target language models derived from unannotated target corpora, combined with a…
Let L be an infinite regular language on a totally ordered alphabet (A,<). Feeding a finite deterministic automaton (with output) with the words of L enumerated lexicographically with respect to < leads to an infinite sequence over the…