English
Related papers

Related papers: On Undetected Redundancy in the Burrows-Wheeler Tr…

200 papers

The Burrows-Wheeler transform (BWT) is a string transformation that enhances string indexing and compressibility. Cotumaccio and Prezza [SODA '21] extended this transformation to nondeterministic finite automata (NFAs) through…

Data Structures and Algorithms · Computer Science 2025-03-11 Ruben Becker , Nicola Cotumaccio , Sung-Hwan Kim , Nicola Prezza , Carlo Tosoni

Motivation: Burrows-Wheeler Transform (BWT) is a common component in full-text indices. Initially developed for data compression, it is particularly powerful for encoding redundant sequences such as pangenome data. However, BWT construction…

Genomics · Quantitative Biology 2025-09-10 Heng Li

In this paper we describe algorithms for computing the BWT and for building (compressed) indexes in external memory. The innovative feature of our algorithms is that they are lightweight in the sense that, for an input of size $n$, they use…

Data Structures and Algorithms · Computer Science 2009-09-25 Paolo Ferragina , Travis Gagie , Giovanni Manzini

Bearing data compression is vital to manage the large volumes of data generated during condition monitoring. In this paper, a novel asymmetrical autoencoder with a lifting wavelet transform (LWT) layer is developed to compress bearing…

Signal Processing · Electrical Eng. & Systems 2025-01-22 Xin Zhu , Ahmet Enis Cetin

We study how the application of injective morphisms affects the number $r$ of equal-letter runs in the Burrows-Wheeler Transform (BWT). This parameter has emerged as a key repetitiveness measure in compressed indexing. We focus on the…

Formal Languages and Automata Theory · Computer Science 2025-04-25 Gabriele Fici , Giuseppe Romana , Marinella Sciortino , Cristian Urbina

We show how to merge two run-length compressed Burrows-Wheeler Transforms (RLBWTs) into a run-length compressed extended Burrows-Wheeler Transform (eBWT) in $O (r)$ space and $O ((r + L) \log (m + n))$ time, where $m$ and $n$ are the…

Data Structures and Algorithms · Computer Science 2026-04-16 Travis Gagie

Indexing highly repetitive strings (i.e., strings with many repetitions) for fast queries has become a central research topic in string processing, because it has a wide variety of applications in bioinformatics and natural language…

Data Structures and Algorithms · Computer Science 2021-04-19 Takaaki Nishimoto , Yasuo Tabei

Constructing the Burrows-Wheeler transform (BWT) for long strings poses significant challenges regarding construction time and memory usage. We use a prefix of the suffix array to partition a long string into shorter substrings, thereby…

Data Structures and Algorithms · Computer Science 2025-05-15 Enno Adler , Stefan Böttcher , Rita Hartel

A standard format used for storing the output of high-throughput sequencing experiments is the FASTQ format. It comprises three main components: (i) headers, (ii) bases (nucleotide sequences), and (iii) quality scores. FASTQ files are…

Data Structures and Algorithms · Computer Science 2023-04-19 Veronica Guerrini , Felipe A. Louza , Giovanna Rosone

Transformer-based models, exemplified by GPT-3, ChatGPT, and GPT-4, have recently garnered considerable attention in both academia and industry due to their promising performance in general language tasks. Nevertheless, these models…

Computation and Language · Computer Science 2023-09-19 Gaochen Dong , Wei Chen

Compressed suffix arrays (CSAs) index large repetitive collections and are key in many text applications. The r-index and its derivatives combine the run-length Burrows-Wheeler Transform (BWT) with suffix array sampling to achieve space…

Data Structures and Algorithms · Computer Science 2026-02-20 Diego Díaz-Domínguez , Veli Mäkinen

The sort transform (ST) is a modification of the Burrows-Wheeler transform (BWT). Both transformations map an arbitrary word of length n to a pair consisting of a word of length n and an index between 1 and n. The BWT sorts all rotation…

Data Structures and Algorithms · Computer Science 2009-08-04 Manfred Kufleitner

Until recently, most experts would probably have agreed we cannot backwards-step in constant time with a run-length compressed Burrows-Wheeler Transform (RLBWT), since doing so relies on rank queries on sparse bitvectors and those inherit…

Data Structures and Algorithms · Computer Science 2022-07-15 Nathaniel K. Brown , Travis Gagie , Massimiliano Rossi

Parameterized strings are a generalization of strings in that their characters are drawn from two different alphabets, where one is considered to be the alphabet of static characters and the other to be the alphabet of parameter characters.…

Data Structures and Algorithms · Computer Science 2022-09-01 Daiki Hashimoto , Diptarama Hendrian , Dominik Köppl , Ryo Yoshinaka , Ayumi Shinohara

When building Burrows-Wheeler Transforms (BWTs) of truly huge datasets, prefix-free parsing (PFP) can use an unreasonable amount of memory. In this paper we show how if a dataset can be broken down into small datasets that are not very…

Data Structures and Algorithms · Computer Science 2025-06-09 Diego Diaz-Dominguez , Travis Gagie , Veronica Guerrini , Ben Langmead , Zsuzsanna Liptak , Giovanni Manzini , Francesco Masillo , Vikram Shivakumar

We survey the different methods used for extending the BWT to collections of strings, following largely [Cenzato and Lipt\'ak, CPM 2022, Bioinformatics 2024]. We analyze the specific aspects and combinatorial properties of the resulting BWT…

Data Structures and Algorithms · Computer Science 2025-06-03 Davide Cenzato , Zsuzsanna Lipták , Nadia Pisanti , Giovanna Rosone , Marinella Sciortino

The Lempel-Ziv factorization (LZ77) and the Run-Length encoded Burrows-Wheeler Transform (RLBWT) are two important tools in text compression and indexing, being their sizes $z$ and $r$ closely related to the amount of text…

Data Structures and Algorithms · Computer Science 2017-02-07 Alberto Policriti , Nicola Prezza

With the popularity of the recent Transformer-based models represented by BERT, GPT-3 and ChatGPT, there has been state-of-the-art performance in a range of natural language processing tasks. However, the massive computations, huge memory…

Computation and Language · Computer Science 2023-04-04 Gaochen Dong , Wei Chen

In this article we extend the elegant in-place Burrows-Wheeler transform (BWT) algorithm proposed by Crochemore et al. (Crochemore et al., 2015). Our extension is twofold: we first show how to compute simultaneously the longest common…

Data Structures and Algorithms · Computer Science 2018-06-08 Felipe A. Louza , Travis Gagie , Guilherme P. Telles

The Positional Burrows--Wheeler Transform (PBWT) is a data structure designed for efficiently representing and querying large collections of sequences, such as haplotype panels in genomics. Forward and backward stepping operations --…

Data Structures and Algorithms · Computer Science 2026-02-17 Paola Bonizzoni , Davide Cozzi , Younan Gao