Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding
Abstract
Large language models (LLMs) face low hardware efficiency during decoding, especially for long-context reasoning tasks. This paper introduces Step-3, a 321B-parameter VLM with hardware-aware model-system co-design optimized for minimizing decoding costs. Step-3 innovates in two key dimensions: (1) A novel Multi-Matrix Factorization Attention (MFA) mechanism that significantly reduces both KV cache size and computation while maintaining high attention expressiveness, and (2) Attention-FFN Disaggregation (AFD), a distributed inference system that decouples attention and Feed-Forward Network (FFN) layers into specialized subsystems. This co-design achieves unprecedented cost efficiency: Step-3 significantly reduces theoretical decoding costs compared with models like DeepSeek-V3 and Qwen3 MoE 235B, with the gains widening at longer context. Step-3 achieves low cost while activating 38B parameters per token (more than DeepSeek-V3 and Qwen3 MoE 235B), demonstrating that hardware-aligned attention arithmetic intensity, MoE sparsity, and AFD are critical to cost-effectiveness. We perform a head-to-head comparison with DeepSeek-V3 in its favorable scenarios. Our implementation on Hopper GPUs achieves a decoding throughput of up to 4,039 tokens per second per GPU under 50ms TPOT SLA (4K context, FP8, no MTP). It is higher than DeepSeek-V3's 2,324 in the same setup and sets a new Pareto frontier for LLM decoding.
Cite
@article{arxiv.2507.19427,
title = {Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding},
author = {StepFun and : and Bin Wang and Bojun Wang and Changyi Wan and Guanzhe Huang and Hanpeng Hu and Haonan Jia and Hao Nie and Mingliang Li and Nuo Chen and Siyu Chen and Song Yuan and Wuxun Xie and Xiaoniu Song and Xing Chen and Xingping Yang and Xuelin Zhang and Yanbo Yu and Yaoyu Wang and Yibo Zhu and Yimin Jiang and Yu Zhou and Yuanwei Lu and Houyi Li and Jingcheng Hu and Ka Man Lo and Ailin Huang and Binxing Jiao and Bo Li and Boyu Chen and Changxin Miao and Chang Lou and Chen Hu and Chen Xu and Chenfeng Yu and Chengyuan Yao and Daokuan Lv and Dapeng Shi and Deshan Sun and Ding Huang and Dingyuan Hu and Dongqing Pang and Enle Liu and Fajie Zhang and Fanqi Wan and Gulin Yan and Han Zhang and Han Zhou and Hanghao Wu and Hangyu Guo and Hanqi Chen and Hanshan Zhang and Hao Wu and Haocheng Zhang and Haolong Yan and Haoran Lv and Haoran Wei and Hebin Zhou and Heng Wang and Heng Wang and Hongxin Li and Hongyu Zhou and Hongyuan Wang and Huiyong Guo and Jia Wang and Jiahao Gong and Jialing Xie and Jian Zhou and Jianjian Sun and Jiaoren Wu and Jiaran Zhang and Jiayu Liu and Jie Cheng and Jie Luo and Jie Yan and Jie Yang and Jieyi Hou and Jinguang Zhang and Jinlan Cao and Jisheng Yin and Junfeng Liu and Junhao Huang and Junzhe Lin and Kaijun Tan and Kaixiang Li and Kang An and Kangheng Lin and Kenkun Liu and Lei Yang and Liang Zhao and Liangyu Chen and Lieyu Shi and Liguo Tan and Lin Lin and Lin Zhang and Lina Chen and Liwen Huang and Liying Shi and Longlong Gu and Mei Chen and Mengqiang Ren and Ming Li and Mingzhe Chen and Na Wang and Nan Wu and Qi Han and Qian Zhao and Qiang Zhang and Qianni Liu and Qiaohui Chen and Qiling Wu and Qinglin He and Qinyuan Tan and Qiufeng Wang and Qiuping Wu and Qiuyan Liang and Quan Sun and Rui Li and Ruihang Miao and Ruosi Wan and Ruyan Guo and Shangwu Zhong and Shaoliang Pang and Shengjie Fan and Shijie Shang and Shilei Jiang and Shiliang Yang and Shiming Hao and Shuli Gao and Siming Huang and Siqi Liu and Tiancheng Cao and Tianhao Cheng and Tianhao Peng and Wang You and Wei Ji and Wen Sun and Wenjin Deng and Wenqing He and Wenzhen Zheng and Xi Chen and Xiangwen Kong and Xianzhen Luo and Xiaobo Yang and Xiaojia Liu and Xiaoxiao Ren and Xin Han and Xin Li and Xin Wu and Xu Zhao and Yanan Wei and Yang Li and Yangguang Li and Yangshijie Xu and Yanming Xu and Yaqiang Shi and Yeqing Shen and Yi Yang and Yifei Yang and Yifeng Gong and Yihan Chen and Yijing Yang and Yinmin Zhang and Yizhuang Zhou and Yuanhao Ding and Yuantao Fan and Yuanzhen Yang and Yuchu Luo and Yue Peng and Yufan Lu and Yuhang Deng and Yuhe Yin and Yujie Liu and Yukun Chen and Yuling Zhao and Yun Mou and Yunlong Li and Yunzhou Ju and Yusheng Li and Yuxiang Yang and Yuxiang Zhang and Yuyang Chen and Zejia Weng and Zhe Xie and Zheng Ge and Zheng Gong and Zhenyi Lu and Zhewei Huang and Zhichao Chang and Zhiguo Huang and Zhirui Wang and Zidong Yang and Zili Wang and Ziqi Wang and Zixin Zhang and Binxing Jiao and Daxin Jiang and Heung-Yeung Shum and Xiangyu Zhang},
journal= {arXiv preprint arXiv:2507.19427},
year = {2025}
}