English
Related papers

Related papers: Tokenisation over Bounded Alphabets is Hard

200 papers

It is well known that sparse approximation problem is \textsf{NP}-hard under general dictionaries. Several algorithms have been devised and analyzed in the past decade under various assumptions on the \emph{coherence} $\mu$ of the…

Computational Complexity · Computer Science 2017-02-10 Ali Çivril

We consider the hardness of approximation of optimization problems from the point of view of definability. For many NP-hard optimization problems it is known that, unless P = NP, no polynomial-time algorithm can give an approximate solution…

Logic in Computer Science · Computer Science 2019-08-30 Albert Atserias , Anuj Dawar

First of all we give some reasons that "natural proofs" built not a barrier to prove P $\not=$ NP using Boolean complexity. Then we investigate the approximation method for its extension to prove super-polynomial lower bounds for the…

Computational Complexity · Computer Science 2020-06-16 Norbert Blum

In this paper we study the {\it bilinear assignment problem} (BAP) with size parameters $m$ and $n$, $m\leq n$. BAP is a generalization of the well known quadratic assignment problem and the three dimensional assignment problem and hence…

Optimization and Control · Mathematics 2016-05-25 Ante Ćustić , Vladyslav Sokol , Abraham P. Punnen , Binay Bhattacharya

In coding and information theory, it is desirable to construct maximal codes that can be either variable length codes or error control codes of fixed length. However deciding code maximality boils down to deciding whether a given NFA is…

Formal Languages and Automata Theory · Computer Science 2022-04-13 Stavros Konstantinidis , Mitja Mastnak , Nelma Moreira , Rogério Reis

Tokenization plays a pivotal role in multilingual NLP. However, existing tokenizers are often skewed towards high-resource languages, limiting their effectiveness for linguistically diverse and morphologically rich languages such as those…

Computation and Language · Computer Science 2025-06-25 N J Karthika , Maharaj Brahma , Rohit Saluja , Ganesh Ramakrishnan , Maunendra Sankar Desarkar

We propose task-adaptive tokenization as a way to adapt the generation pipeline to the specifics of a downstream task and enhance long-form generation in mental health. Inspired by insights from cognitive science, our task-adaptive…

Computation and Language · Computer Science 2023-11-14 Siyang Liu , Naihao Deng , Sahand Sabour , Yilin Jia , Minlie Huang , Rada Mihalcea

The seminar assignment problem is a variant of the generalized assignment problem in which items have unit size and the amount of space allowed in each bin is restricted to an arbitrary set of values. The problem has been shown to be…

Data Structures and Algorithms · Computer Science 2016-10-18 Amotz Bar-Noy , George Rabanca

Subword tokenization segments input text according to a pre-defined vocabulary to feed it into a language model; the language model, in turn, generates a sequence made from this same vocabulary. The members of the vocabulary can be built of…

Computation and Language · Computer Science 2025-11-11 Preston Firestone , Shubham Ugare , Gagandeep Singh , Sasa Misailovic

We consider Parikh images of languages accepted by non-deterministic finite automata and context-free grammars; in other words, we treat the languages in a commutative way --- we do not care about the order of letters in the accepted word,…

Formal Languages and Automata Theory · Computer Science 2010-03-23 Eryk Kopczyński

We investigate fine-grained algorithmic aspects of identification problems in graphs and set systems, with a focus on Locating-Dominating Set and Test Cover. We prove the (tight) conditional lower bounds for these problems when…

Data Structures and Algorithms · Computer Science 2025-08-25 Dipayan Chakraborty , Florent Foucaud , Diptapriyo Majumdar , Prafullkumar Tale

Optimization problems consist of either maximizing or minimizing an objective function. Instead of looking for a maximum solution (resp. minimum solution), one can find a minimum maximal solution (resp. maximum minimal solution). Such…

Data Structures and Algorithms · Computer Science 2018-11-08 Kaveh Khoshkhah , Mehdi Khosravian Ghadikolaei , Jerome Monnot , Florian Sikora

We study extremal and algorithmic questions of subset and careful synchronization in monotonic automata. We show that several synchronization problems that are hard in general automata can be solved in polynomial time in monotonic automata,…

Formal Languages and Automata Theory · Computer Science 2017-11-27 Andrew Ryzhikov , Anton Shemyakov

The field of kernelization studies polynomial-time preprocessing routines for hard problems in the framework of parameterized complexity. Although a framework for proving kernelization lower bounds has been discovered in 2008 and…

Data Structures and Algorithms · Computer Science 2011-11-03 Marek Cygan , Stefan Kratsch , Marcin Pilipczuk , Michał Pilipczuk , Magnus Wahlström

Constraint Satisfaction Problem on finite sets is known to be NP-complete in general but certain restrictions on the constraint language can ensure tractability. It was proved that if a constraint language has a weak near unanimity…

Computational Complexity · Computer Science 2019-03-07 Dmitriy Zhuk

Indexing a set of strings for prefix search or membership queries is a fundamental task with many applications such as information retrieval or database systems. A classic abstract data type for modelling such an index is a trie. Due to the…

Data Structures and Algorithms · Computer Science 2024-03-11 Hideo Bannai , Keisuke Goto , Shunsuke Kanda , Dominik Köppl

The polarization process of polar codes over a ternary alphabet is studied. Recently it has been shown that the scaling of the blocklength of polar codes with prime alphabet size scales polynomially with respect to the inverse of the gap…

Information Theory · Computer Science 2015-02-11 Dina Goldin , David Burshtein

Set Cover is a classic NP-hard problem; as shown by Slav\'{i}k (1997) the greedy algorithm gives an approximation ratio of $\ln n - \ln \ln n + \Theta(1)$. A series of works by Lund \& Yannakakis (1994), Feige (1998), Moshkovitz (2015) have…

Data Structures and Algorithms · Computer Science 2017-02-17 David G. Harris

An alphabetic binary tree formulation applies to problems in which an outcome needs to be determined via alphabetically ordered search prior to the termination of some window of opportunity. Rather than finding a decision tree minimizing…

Information Theory · Computer Science 2009-03-28 Michael B. Baer

Mixed codes, which are error-correcting codes in the Cartesian product of different-sized spaces, model degrading storage systems well. While such codes have previously been studied for their algebraic properties (e.g., existence of perfect…

Information Theory · Computer Science 2022-12-20 Yonatan Yehezkeally , Haider Al Kim , Sven Puchinger , Antonia Wachter-Zeh