English
Related papers

Related papers: Communication-Inspired Tokenization for Structured…

200 papers

We introduce CompTok, a training framework for learning visual tokenizers whose tokens are enhanced for compositionality. CompTok uses a token-conditioned diffusion decoder. By employing an InfoGAN-style objective, where we train a…

Computer Vision and Pattern Recognition · Computer Science 2026-02-04 Bingchen Zhao , Qiushan Guo , Ye Wang , Yixuan Huang , Zhonghua Zhai , Yu Tian

This paper proposes new framework of communication system leveraging promising generation capabilities of multi-modal generative models. Regarding nowadays smart applications, successful communication can be made by conveying the perceptual…

Signal Processing · Electrical Eng. & Systems 2023-09-11 Hyelin Nam , Jihong Park , Jinho Choi , Seong-Lyun Kim

Foundation models in language and vision benefit from a unified discrete token interface that converts raw inputs into sequences for scalable pre-training and inference. For graphs, an effective tokenizer should yield reusable discrete…

Information Retrieval · Computer Science 2026-05-28 Yang Xiang , Li Fan , Chenke Yin , Lutz Oettershagen , Chengtao Ji

Recent advances in explainable recommendations have explored the integration of language models to analyze natural language rationales for user-item interactions. Despite their potential, existing methods often rely on ID-based…

Machine Learning · Computer Science 2025-12-18 Xinshun Feng , Mingzhe Liu , Yi Qiao , Tongyu Zhu , Leilei Sun , Shuai Wang

Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap. However, discretization inevitably discards fine-grained semantic information, leading to suboptimal performance in visual…

Computer Vision and Pattern Recognition · Computer Science 2026-03-12 Yaqi Zhao , Wang Lin , Zijian Zhang , Miles Yang , Jingyuan Chen , Wentao Zhang , Zhao Zhong , Liefeng Bo

Tokens or patches within Vision Transformers (ViT) lack essential semantic information, unlike their counterparts in natural language processing (NLP). Typically, ViT tokens are associated with rectangular image patches that lack specific…

Computer Vision and Pattern Recognition · Computer Science 2024-02-29 Young Kyung Kim , J. Matías Di Martino , Guillermo Sapiro

Self-attention is central to the success of Transformer architectures; however, learning the query, key, and value projections from random initialization remains challenging and computationally expensive. In this paper, we propose two…

Computer Vision and Pattern Recognition · Computer Science 2026-05-18 Hongyi Pan , Emadeldeen Hamdan , Xin Zhu , Ahmet Enis Cetin , Ulas Bagci

Image tokenization, the process of transforming raw image pixels into a compact low-dimensional latent representation, has proven crucial for scalable and efficient image generation. However, mainstream image tokenization methods generally…

Computer Vision and Pattern Recognition · Computer Science 2025-04-08 Kaiwen Zha , Lijun Yu , Alireza Fathi , David A. Ross , Cordelia Schmid , Dina Katabi , Xiuye Gu

Modern multimodal large language models (MLLMs) typically keep the language model fixed and train a visual projector that maps the pixels into a sequence of tokens in its embedding space, so that images can be presented in essentially the…

Computer Vision and Pattern Recognition · Computer Science 2026-05-19 Hyun Lee , Hyemin Jeong , Yejin Kim , Hyungwook Choi , Hyunsoo Cho , Soo Kyung Kim , Joonseok Lee

Chain-of-thought (CoT) decoding enables language models to improve reasoning performance at the cost of high generation latency in decoding. Recent proposals have explored variants of contemplation tokens, a term we introduce that refers to…

Computation and Language · Computer Science 2024-12-18 Jeffrey Cheng , Benjamin Van Durme

Existing vision tokenization isolates the optimization of vision tokenizers from downstream training, implicitly assuming the visual tokens can generalize well across various tasks, e.g., image generation and visual question answering. The…

Computer Vision and Pattern Recognition · Computer Science 2025-05-16 Wenxuan Wang , Fan Zhang , Yufeng Cui , Haiwen Diao , Zhuoyan Luo , Huchuan Lu , Jing Liu , Xinlong Wang

We present SceneTok, a novel tokenizer for encoding view sets of scenes into a compressed and diffusable set of unstructured tokens. Existing approaches for 3D scene representation and generation commonly use 3D data structures or…

Computer Vision and Pattern Recognition · Computer Science 2026-03-02 Mohammad Asim , Christopher Wewer , Jan Eric Lenssen

World models provide a powerful framework for simulating environment dynamics conditioned on actions or instructions, enabling downstream tasks such as action planning or policy learning. Recent approaches leverage world models as learned…

Computer Vision and Pattern Recognition · Computer Science 2026-03-06 Dongwon Kim , Gawon Seo , Jinsung Lee , Minsu Cho , Suha Kwak

Typical convolutional networks are trained and conducted on RGB images. However, images are often compressed for memory savings and efficient transmission in real-world applications. In this paper, we explore methods for performing semantic…

Computer Vision and Pattern Recognition · Computer Science 2020-01-01 Shao-Yuan Lo , Hsueh-Ming Hang

Text-to-image (TTI) diffusion models have demonstrated impressive results in generating high-resolution images of complex and imaginative scenes. Recent approaches have further extended these methods with personalization techniques that…

Computer Vision and Pattern Recognition · Computer Science 2025-05-05 Tanzila Rahman , Shweta Mahajan , Hsin-Ying Lee , Jian Ren , Sergey Tulyakov , Leonid Sigal

Token representations influence the efficiency and adaptability of language models, yet conventional tokenization strategies impose rigid segmentation boundaries that do not adjust dynamically to evolving contextual relationships. The…

Computation and Language · Computer Science 2025-08-11 Alistair Dombrowski , Beatrix Engelhardt , Dimitri Fairbrother , Henry Evidail

Adaptive cognition requires structured internal models of objects and their relations. Predictive neural networks are often proposed to learn such world models, but how these are instantiated and how they support prediction remain unclear.…

Machine Learning · Computer Science 2026-05-11 Linda Ariel Ventura , Victoria Bosch , Tim C Kietzmann , Sushrut Thorat

In language processing, transformers benefit greatly from text being condensed. This is achieved through a larger vocabulary that captures word fragments instead of plain characters. This is often done with Byte Pair Encoding. In the…

Computer Vision and Pattern Recognition · Computer Science 2024-11-18 Tim Elsner , Paula Usinger , Julius Nehring-Wirxel , Gregor Kobsik , Victor Czech , Yanjiang He , Isaak Lim , Leif Kobbelt

Tokens are becoming the basic units through which foundation models represent and process information for understanding and inference. However, traditional wireless communication, centered on bit-level fidelity, faces a mismatch between…

Machine Learning · Computer Science 2026-05-22 Sige Liu , Kezhi Wang

Modern change detection (CD) has achieved remarkable success by the powerful discriminative ability of deep convolutions. However, high-resolution remote sensing CD remains challenging due to the complexity of objects in the scene. Objects…

Computer Vision and Pattern Recognition · Computer Science 2021-07-13 Hao Chen , Zipeng Qi , Zhenwei Shi