相关论文: Overhead-Free Computation, DCFLs, and CFLs

A major target of linguistics and cognitive science has been to understand what class of learning systems can acquire the key structures of natural language. Until recently, the computational requirements of language have been used to argue…

A language is dense if the set of all infixes (or subwords) of the language is the set of all words. Here, it is shown that it is decidable whether the language accepted by a nondeterministic Turing machine with a one-way read-only input…

At first glance, one-state Turing machines are very weak: the halting problem for them is decidable, and, without memory, they cannot even accept a simple one element language such as $L = \{ 1 \}$ . Nevertheless it has been showed that a…

We discuss the computational complexity of context-free languages, concentrating on two well-known structural properties---immunity and pseudorandomness. An infinite language is REG-immune (resp., CFL-immune) if it contains no infinite…

LS is a particular type of computational processes simulating living tissue. They use an unlimited branching process arising from the simultaneous substitutions of some words instead of letters in some initial word. This combines the…

Using nonstandard analysis, we will extend the classical Turing machines into the internal Turing machines. The internal Turing machines have the capability to work with infinite ($*$-finite) number of bits while keeping the finite…

Understanding how the structure of language can be learned from sentences alone is a central question in both cognitive science and machine learning. Studies of the internal representations of Large Language Models (LLMs) support their…

Polynomial--time constant--space quantum Turing machines (QTMs) and logarithmic--space probabilistic Turing machines (PTMs) recognize uncountably many languages with bounded error (Say and Yakary\i lmaz 2014, arXiv:1411.7647). In this…

Recent work on tokenizer-free multilingual pretrained models show promising results in improving cross-lingual transfer and reducing engineering overhead (Clark et al., 2022; Xue et al., 2022). However, these works mainly focus on reporting…

We present a semi-automated framework to construct and reason about programs in a deeply-embedded while-language. The while-language we consider is a simple computation model that can simulate (and be simulated by) Turing Machines with a…

We observe that pre-trained large language models (LLMs) are capable of autoregressively completing complex token sequences -- from arbitrary ones procedurally generated by probabilistic context-free grammars (PCFG), to more rich spatial…

To expand a fundamental theory of context-free languages, we equip nondeterministic one-way pushdown automata with additional oracle mechanisms, which naturally induce various nondeterministic reducibilities among formal languages. As a…

Each step that results in a bit of information being ``forgotten'' by a computing device has an intrinsic energy cost. Although any Turing machine can be rewritten to be thermodynamically reversible without changing the recognized language,…

The paper is about a class of languages that extends context-free languages (CFL) and is stable under shuffle. Specifically, we investigate the class of partially-commutative context-free languages (PCCFL), where non-terminal symbols are…

Turing machines define polynomial time (PTime) on strings but cannot deal with structures like graphs directly, and there is no known, easily computable string encoding of isomorphism classes of structures. Is there a computation model…

In this article we prove that the general transformer neural model undergirding modern large language models (LLMs) is Turing complete under reasonable assumptions. This is the first work to directly address the Turing completeness of the…

We show that transformer-based large language models are computationally universal when augmented with an external memory. Any deterministic language model that conditions on strings of bounded length is equivalent to a finite automaton,…

Probabilistic context-free grammars (PCFGs) with neural parameterization have been shown to be effective in unsupervised phrase-structure grammar induction. However, due to the cubic computational complexity of PCFG representation and…

We analyze the computational power of discrete-time recurrent neural networks (NNs) with the saturated-linear activation function within the Chomsky hierarchy. This model restricted to integer weights coincides with binary-state NNs with…

Valence automata are a generalization of various models of automata with storage. Here, each edge carries, in addition to an input word, an element of a monoid. A computation is considered valid if multiplying the monoid elements on the…