Computation and Language · Computer Science
Analyzing Feed-Forward Blocks in Transformers through the Lens of Attention Maps
Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, Kentaro Inui
2024-04-16
Computer Vision and Pattern Recognition · Computer Science
Skip-Vision: Efficient and Scalable Acceleration of Vision-Language Models via Adaptive Token Skipping
Weili Zeng, Ziyuan Huang, Kaixiang Ji, Yichao Yan
2025-07-04
Machine Learning · Computer Science
Federated Learning for Inference at Anytime and Anywhere
Zicheng Liu, Da Li, Javier Fernandez-Marques, Stefanos Laskaridis +5
2022-12-09
Computation and Language · Computer Science
Reasoning is Periodicity? Improving Large Language Models Through Effective Periodicity Modeling
Yihong Dong, Ge Li, Xue Jiang, Yongding Tao +7
2025-10-28
Computation and Language · Computer Science
Fast Byte Latent Transformer
Julie Kallini, Artidoro Pagnoni, Tomasz Limisiewicz, Gargi Ghosh +4
2026-05-11
Computation and Language · Computer Science
Fovea Transformer: Efficient Long-Context Modeling with Structured Fine-to-Coarse Attention
Ziwei He, Jian Yuan, Le Zhou, Jingwen Leng +1
2024-01-12
Computation and Language · Computer Science
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space
Mor Geva, Avi Caciularu, Kevin Ro Wang, Yoav Goldberg
2022-10-14
Distributed, Parallel, and Cluster Computing · Computer Science
Large Language Model Partitioning for Low-Latency Inference at the Edge
Dimitrios Kafetzis, Ramin Khalili, Iordanis Koutsopoulos
2025-05-06
Machine Learning · Computer Science
Fast Forward: Accelerating LLM Prefill with Predictive FFN Sparsity
Aayush Gautam, Mukul Gagrani, Junyoung Park, Mingu Lee +2
2026-02-03
Computation and Language · Computer Science
Cerberus: Efficient Inference with Adaptive Parallel Decoding and Sequential Knowledge Enhancement
Yuxuan Liu, Wenyuan Li, Laizhong Cui, Hailiang Yang
2024-10-18
Distributed, Parallel, and Cluster Computing · Computer Science
FTRANS: Energy-Efficient Acceleration of Transformers using FPGA
Bingbing Li, Santosh Pandey, Haowen Fang, Yanjun Lyv +6
2020-07-20
Computation and Language · Computer Science
KunlunBaize: LLM with Multi-Scale Convolution and Multi-Token Prediction Under TransformerX Framework
Cheng Li, Jiexiong Liu, Yixuan Chen, Yanqin Jia +1
2025-03-21
Machine Learning · Computer Science
Thoughtbubbles: an Unsupervised Method for Parallel Thinking in Latent Space
Houjun Liu, Shikhar Murty, Christopher D. Manning, Róbert Csordás
2026-02-02
Computation and Language · Computer Science
Building on Efficient Foundations: Effectively Training LLMs with Structured Feedforward Layers
Xiuying Wei, Skander Moalla, Razvan Pascanu, Caglar Gulcehre
2024-11-07
Computation and Language · Computer Science
Fusing Large Language Models with Temporal Transformers for Time Series Forecasting
Chen Su, Yuanhe Tian, Qinyu Liu, Jun Zhang +1
2025-07-15
Computation and Language · Computer Science
Probe and Skip: Self-Predictive Token Skipping for Efficient Long-Context LLM Inference
Zimeng Wu, Donghao Wang, Chaozhe Jin, Jiaxin Chen +1
2026-02-03
Computation and Language · Computer Science
ChunkLLM: A Lightweight Pluggable Framework for Accelerating LLMs Inference
Haojie Ouyang, Jianwei Lv, Lei Ren, Chen Wei +2
2026-05-26
Computation and Language · Computer Science
Bayesian Transformer Language Models for Speech Recognition
Boyang Xue, Jianwei Yu, Junhao Xu, Shansong Liu +5
2021-02-10
Computation and Language · Computer Science
SPEED: Speculative Pipelined Execution for Efficient Decoding
Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Hasan Genc +3
2024-01-04
Computation and Language · Computer Science
FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping
Ajay Jaiswal, Bodun Hu, Lu Yin, Yeonju Ro +3
2024-04-08
Distributed, Parallel, and Cluster Computing · Computer Science
SplitLLM: Collaborative Inference of LLMs for Model Placement and Throughput Optimization
Akrit Mudvari, Yuang Jiang, Leandros Tassiulas
2024-10-17
Computation and Language · Computer Science
Meta-Learning Fast Weight Language Models
Kevin Clark, Kelvin Guu, Ming-Wei Chang, Panupong Pasupat +2
2022-12-06