English
Related papers

Related papers: (1D) Ordered Tokens Enable Efficient Test-Time Sea…

200 papers

Recently presented Token-Oriented Object Notation (TOON) aims to replace JSON as a serialization format for passing structured data to LLMs with significantly reduced token usage. While showing solid accuracy in LLM comprehension, there is…

Computation and Language · Computer Science 2026-03-05 Ivan Matveev

Autoregressive (AR) models have recently shown strong performance in image generation, where a critical component is the visual tokenizer (VT) that maps continuous pixel inputs to discrete token sequences. The quality of the VT largely…

Computer Vision and Pattern Recognition · Computer Science 2025-05-20 Huawei Lin , Tong Geng , Zhaozhuo Xu , Weijie Zhao

Discrete image tokenizers are commonly trained in two stages: first for reconstruction, and then with a prior model fitted to the frozen token sequences. This decoupling leaves the tokenizer unaware of the model that will later generate its…

Computer Vision and Pattern Recognition · Computer Science 2026-05-08 Bowen Zheng , Yihong Luo , Tianyang Hu

In the domain of image generation, latent-based generative models occupy a dominant status; however, these models rely heavily on image tokenizer. To meet modeling requirements, autoregressive models possessing the characteristics of…

Computer Vision and Pattern Recognition · Computer Science 2025-03-24 Panpan Wang , Liqiang Niu , Fandong Meng , Jinan Xu , Yufeng Chen , Jie Zhou

Autoregressive models have emerged as a powerful approach for visual generation but suffer from slow inference speed due to their sequential token-by-token prediction process. In this paper, we propose a simple yet effective approach for…

Computer Vision and Pattern Recognition · Computer Science 2025-04-04 Yuqing Wang , Shuhuai Ren , Zhijie Lin , Yujin Han , Haoyuan Guo , Zhenheng Yang , Difan Zou , Jiashi Feng , Xihui Liu

Generative recommendation (GR) models generate items by autoregressively producing a sequence of discrete tokens that jointly index the target item. However, this autoregressive generation process also induces a structured decoding space…

Information Retrieval · Computer Science 2026-05-08 Yupeng Hou , Haven Kim , Clark Mingxuan Ju , Eduardo Escoto , Neil Shah , Julian McAuley

We propose task-adaptive tokenization as a way to adapt the generation pipeline to the specifics of a downstream task and enhance long-form generation in mental health. Inspired by insights from cognitive science, our task-adaptive…

Computation and Language · Computer Science 2023-11-14 Siyang Liu , Naihao Deng , Sahand Sabour , Yilin Jia , Minlie Huang , Rada Mihalcea

Recent advances in image tokenizers, such as VQ-VAE, have enabled text-to-image generation using auto-regressive methods, similar to language modeling. However, these methods have yet to leverage pre-trained language models, despite their…

Computer Vision and Pattern Recognition · Computer Science 2024-09-26 Yuhui Zhang , Brandon McKinzie , Zhe Gan , Vaishaal Shankar , Alexander Toshev

Autoregressive (AR) models have shown great promise in image generation, yet they face a fundamental inefficiency stemming from their core component: a vast, unstructured vocabulary of visual tokens. This conventional approach treats tokens…

Computer Vision and Pattern Recognition · Computer Science 2025-10-07 Lixuan He , Shikang Zheng , Linfeng Zhang

In image-based camera localization systems, information about the environment is usually stored in some representation, which can be referred to as a map. Conventionally, most maps are built upon hand-crafted features. Recently, neural…

Computer Vision and Pattern Recognition · Computer Science 2019-04-17 Mingpan Guo , Stefan Matthes , Jiaojiao Ye , Hao Shen

While diffusion models dominate the field of visual generation, they are computationally inefficient, applying a uniform computational effort regardless of different complexity. In contrast, autoregressive (AR) models are inherently…

Computer Vision and Pattern Recognition · Computer Science 2026-04-15 Jian Han , Jinlai Liu , Jiahuan Wang , Bingyue Peng , Zehuan Yuan

Modeling genomic sequences faces two unsolved challenges: the information density varies widely across different regions, while there is no clearly defined minimum vocabulary unit. Relying on either four primitive bases or independently…

Genomics · Quantitative Biology 2025-11-20 Siyuan Li , Kai Yu , Anna Wang , Zicheng Liu , Chang Yu , Jingbo Zhou , Qirong Yang , Yucheng Guo , Xiaoming Zhang , Stan Z. Li

Generative recommendation models can model user behavior as sequences of events and provide a shared backbone for multiple recommendation tasks. In production, however, pre-training gains do not automatically translate into downstream…

Information Retrieval · Computer Science 2026-05-25 Qiuling Xu , Ko-Jen Hsiao , Moumita Bhattacharya

Autoregressive (AR) video generative models rely on video tokenizers that compress pixels into discrete token sequences. The length of these token sequences is crucial for balancing reconstruction quality against downstream generation…

Computer Vision and Pattern Recognition · Computer Science 2026-03-13 Tianwei Xiong , Jun Hao Liew , Zilong Huang , Zhijie Lin , Jiashi Feng , Xihui Liu

Adaptive cognition requires structured internal models of objects and their relations. Predictive neural networks are often proposed to learn such world models, but how these are instantiated and how they support prediction remain unclear.…

Machine Learning · Computer Science 2026-05-11 Linda Ariel Ventura , Victoria Bosch , Tim C Kietzmann , Sushrut Thorat

Foundation models have significantly enhanced 2D task performance, and recent works like Bridge3D have successfully applied these models to improve 3D scene understanding through knowledge distillation, marking considerable advancements.…

Computer Vision and Pattern Recognition · Computer Science 2024-10-18 Zhimin Chen , Liang Yang , Yingwei Li , Longlong Jing , Bing Li

Can attention- or gradient-based visualization techniques be used to infer token-level labels for binary sequence tagging problems, using networks trained only on sentence-level labels? We construct a neural network architecture based on…

Computation and Language · Computer Science 2018-05-08 Marek Rei , Anders Søgaard

Efficient Neural Architecture Search methods based on weight sharing have shown good promise in democratizing Neural Architecture Search for computer vision models. There is, however, an ongoing debate whether these efficient methods are…

Machine Learning · Computer Science 2020-08-17 Gabriel Bender , Hanxiao Liu , Bo Chen , Grace Chu , Shuyang Cheng , Pieter-Jan Kindermans , Quoc Le

To parse images into fine-grained semantic parts, the complex fine-grained elements will put it in trouble when using off-the-shelf semantic segmentation networks. In this paper, for image parsing task, we propose to parse images from…

Computer Vision and Pattern Recognition · Computer Science 2018-04-24 Jiagao Hu , Zhengxing Sun , Yunhan Sun , Jinlong Shi

Generative models aim to learn the probability distributions underlying data, enabling the generation of new, realistic samples. Quantum inspired generative models, such as Born machines based on the matrix product state framework, have…

Machine Learning · Computer Science 2025-12-30 Wanda Hou , Miao Li , Yi-Zhuang You