Related papers: A Diamond Structure in the Transducer Hierarchy
We find limits to the Transformer architecture for language modeling and show it has a universal prediction property in an information-theoretic sense. We further analyze performance in non-asymptotic data regimes to understand the role of…
We give a new, and hopefully more easily understandable, structural proof of the decomposition of a $k$-valued transducer into $k$ unambiguous functional ones, a result established by A. Weber in 1996. Our construction is based on a…
Concatenation hierarchies are classifications of regular languages. All such hierarchies are built through the same construction process: start from an initial class of languages and build new levels using two generic operations.…
Predicate logic is the premier choice for specifying classes of relational structures. Homomorphisms are key to describing correspondences between relational structures. Questions concerning the interdependencies between these two means of…
We obtain several results concerning the concept of isotypic structures. Namely we prove that any field of finite transcendence degree over a prime subfield is defined by types; then we construct isotypic but not isomorphic structures with…
The Medvedev degree of a subshift is a dynamical invariant of computable origin that can be used to compare the complexity of subshifts that contain only uncomputable configurations. We develop theory to describe how these degrees can be…
A new class of structured matrices is presented and a closed form formula for their determinant is established. This formula has strong connections with the one for Vandermonde matrices.
Generally, in any human field, a Smarandache Structure on a set A means a weak structure W on A such that there exists a proper subset B which is embedded with a stronger structure S. By proper subset one understands a set included in A,…
As transformers have gained prominence in natural language processing, some researchers have investigated theoretically what problems they can and cannot solve, by treating problems as formal languages. Exploring such questions can help…
This paper initiates the dialectical approach to net theory. This approach views nets as special, but very important and natural, dialectical systems. By following this approach, a suitably generalized version of nets, called dialectical…
A necklace is an equivalence class of words of length $n$ over an alphabet under the cyclic shift (rotation) operation. As a classical object, there have been many algorithmic results for key operations on necklaces, including counting,…
The origin semantics for transducers was proposed in 2014, and led to various characterizations and decidability results that are in contrast with the classical semantics. In this paper we add a further decidability result for…
A fundamental question in interpretability research is to what extent neural networks, particularly language models, implement reusable functions through subnetworks that can be composed to perform more complex tasks. Recent advances in…
The theory of regular and aperiodic transformations of finite strings has recently received a lot of interest. These classes can be equivalently defined using logic (Monadic second-order logic and first-order logic), two-way machines…
We introduce a new combinatorial object called tower diagrams and prove fundamental properties of these objects. We also introduce an algorithm that allows us to slide words to tower diagrams. We show that the algorithm is well-defined only…
This article extends Scholze's six functor formalism for diamonds to a very general class of stacky morphisms between v-stacks, using $\infty$-categorical techniques developed by Liu-Zheng.
Similar to a tree grammar, a Horn theory can be used to describe an infinite set of terms. In this paper, we present a class of Horn theories such that the set of definable predicates is closed wrt. conjunction and such that the…
We begin a systematic development of structure theory for a first order theory, which is stable over a monadic predicate. We show that stability over a predicate implies quantifier free definability of types over stable sets, introduce an…
We prove the existence of a canonical form for semi-deterministic transducers with incomparable sets of output strings. Based on this, we develop an algorithm which learns semi-deterministic transducers given access to translation queries.…
The Transformer architecture has become increasingly popular over the past two years, owing to its impressive performance on a number of natural language processing (NLP) tasks. However, all Transformer computations occur at the level of…