English
Related papers

Related papers: Multidimensional Byte Pair Encoding: Shortened Seq…

200 papers

The Space-Air-Ground-Sea integrated network calls for more robust and secure transmission techniques against jamming. In this paper, we propose a textual semantic transmission framework for robust transmission, which utilizes the advanced…

Computation and Language · Computer Science 2023-07-25 Shicong Liu , Zhen Gao , Gaojie Chen , Yu Su , Lu Peng

Transformers have impressive generalization capabilities on tasks with a fixed context length. However, they fail to generalize to sequences of arbitrary length, even for seemingly simple tasks such as duplicating a string. Moreover, simply…

In this paper, we present TridentSE, a novel architecture for speech enhancement, which is capable of efficiently capturing both global information and local details. TridentSE maintains T-F bin level representation to capture details, and…

Audio and Speech Processing · Electrical Eng. & Systems 2022-10-25 Dacheng Yin , Zhiyuan Zhao , Chuanxin Tang , Zhiwei Xiong , Chong Luo

Vision encoders serve as the cornerstone of multimodal understanding. Single-encoder architectures like CLIP exhibit inherent constraints in generalizing across diverse multimodal tasks, while recent multi-encoder fusion methods introduce…

Computer Vision and Pattern Recognition · Computer Science 2025-07-29 Yuchen Liu , Yaoming Wang , Bowen Shi , Xiaopeng Zhang , Wenrui Dai , Chenglin Li , Hongkai Xiong , Qi Tian

The assumption across nearly all language model (LM) tokenization schemes is that tokens should be subwords, i.e., contained within word boundaries. While providing a seemingly reasonable inductive bias, is this common practice limiting the…

Computation and Language · Computer Science 2025-08-28 Alisa Liu , Jonathan Hayase , Valentin Hofmann , Sewoong Oh , Noah A. Smith , Yejin Choi

Constrained sequence codes have been widely used in modern communication and data storage systems. Sequences encoded with constrained sequence codes satisfy constraints imposed by the physical channel, hence enabling efficient and reliable…

Information Theory · Computer Science 2018-09-07 Congzhe Cao , Duanshun Li , Ivan Fair

Modeling genomic sequences faces two unsolved challenges: the information density varies widely across different regions, while there is no clearly defined minimum vocabulary unit. Relying on either four primitive bases or independently…

Genomics · Quantitative Biology 2025-11-20 Siyuan Li , Kai Yu , Anna Wang , Zicheng Liu , Chang Yu , Jingbo Zhou , Qirong Yang , Yucheng Guo , Xiaoming Zhang , Stan Z. Li

Pre-trained speech Transformers have facilitated great success across various speech processing tasks. However, fine-tuning these encoders for downstream tasks require sufficiently large training data to converge or to achieve…

Computation and Language · Computer Science 2022-10-25 Hao Yang , Jinming Zhao , Gholamreza Haffari , Ehsan Shareghi

Video-based dialog task is a challenging multimodal learning task that has received increasing attention over the past few years with state-of-the-art obtaining new performance records. This progress is largely powered by the adaptation of…

Computer Vision and Pattern Recognition · Computer Science 2022-10-27 Huda Alamri , Anthony Bilic , Michael Hu , Apoorva Beedu , Irfan Essa

Since the introduction of Vision Transformer (ViT), patchification has long been regarded as a de facto image tokenization approach for plain visual architectures. By compressing the spatial size of images, this approach can effectively…

Computer Vision and Pattern Recognition · Computer Science 2026-02-23 Feng Wang , Yaodong Yu , Guoyizhe Wei , Wei Shao , Yuyin Zhou , Alan Yuille , Cihang Xie

Vision Transformers (ViTs) can learn strong image-level representations while their patch representations become less effective for dense prediction during prolonged training. We revisit this dense degradation phenomenon and argue that it…

Computer Vision and Pattern Recognition · Computer Science 2026-05-25 Linxiang Su

Vision Transformers (ViTs) have achieved strong performance in visual recognition, yet their deployment in resource-constrained industrial environments remains limited. Some main challenges are their high computational cost, memory…

Computer Vision and Pattern Recognition · Computer Science 2026-05-05 Phat Nguyen , Xue Geng , Kaixin Xu , Wang Zhe , Xulei Yang , Ngai-Man Cheung

Diffusion models have revolutionized generative tasks, especially in the domain of text-to-image synthesis; however, their iterative denoising process demands substantial computational resources. In this paper, we present a novel…

Computer Vision and Pattern Recognition · Computer Science 2025-02-04 Xinle Cheng , Zhuoming Chen , Zhihao Jia

Tokenization is widely used in large language models because it significantly improves performance. However, tokenization imposes several disadvantages, such as performance biases, increased adversarial vulnerability, decreased…

Computation and Language · Computer Science 2024-10-08 Kevin Slagle

We propose task-adaptive tokenization as a way to adapt the generation pipeline to the specifics of a downstream task and enhance long-form generation in mental health. Inspired by insights from cognitive science, our task-adaptive…

Computation and Language · Computer Science 2023-11-14 Siyang Liu , Naihao Deng , Sahand Sabour , Yilin Jia , Minlie Huang , Rada Mihalcea

Image translation is a computer vision task that involves translating one representation of the scene into another. Various approaches have been proposed and achieved highly desirable results. Nevertheless, its accomplishment requires…

Computer Vision and Pattern Recognition · Computer Science 2018-11-02 Dichao Hu

Standard Byte-Pair Encoding (BPE) tokenization compresses text by pairing a learned token vocabulary with a detailed merge list. Recent work has shown that this merge list exposes a potential attack surface for extracting information about…

Computation and Language · Computer Science 2025-08-12 Tomohiro Sawada , Kartik Goyal

Encoder transformer models compress information from all tokens in a sequence into a single [CLS] token to represent global context. This approach risks diluting fine-grained or hierarchical features, leading to information loss in…

Computation and Language · Computer Science 2025-09-23 Asif Shahriar , Rifat Shahriyar , M Saifur Rahman

In complex visual recognition tasks it is typical to adopt multiple descriptors, that describe different aspects of the images, for obtaining an improved recognition performance. Descriptors that have diverse forms can be fused into a…

Computer Vision and Pattern Recognition · Computer Science 2015-06-15 Jayaraman J. Thiagarajan , Karthikeyan Natesan Ramamurthy , Andreas Spanias

Text-to-video generation poses significant challenges due to the inherent complexity of video data, which spans both temporal and spatial dimensions. It introduces additional redundancy, abrupt variations, and a domain gap between language…

Computer Vision and Pattern Recognition · Computer Science 2025-03-17 Ziqin Zhou , Yifan Yang , Yuqing Yang , Tianyu He , Houwen Peng , Kai Qiu , Qi Dai , Lili Qiu , Chong Luo , Lingqiao Liu