Semantic Search At LinkedIn
Abstract
Semantic search with large language models (LLMs) enables retrieval by meaning rather than keyword overlap, but scaling it requires major inference efficiency advances. We present LinkedIn's LLM-based semantic search framework for AI Job Search and AI People Search, combining an LLM relevance judge, embedding-based retrieval, and a compact Small Language Model trained via multi-teacher distillation to jointly optimize relevance and engagement. A prefill-oriented inference architecture co-designed with model pruning, context compression, and text-embedding hybrid interactions boosts ranking throughput by over 75x under a fixed latency constraint while preserving near-teacher-level NDCG, enabling one of the first production LLM-based ranking systems with efficiency comparable to traditional approaches and delivering significant gains in quality and user engagement.
Cite
@article{arxiv.2602.07309,
title = {Semantic Search At LinkedIn},
author = {Fedor Borisyuk and Sriram Vasudevan and Muchen Wu and Guoyao Li and Benjamin Le and Shaobo Zhang and Qianqi Kay Shen and Yuchin Juan and Kayhan Behdin and Liming Dong and Kaixu Yang and Shusen Jing and Ravi Pothamsetty and Rajat Arora and Sophie Yanying Sheng and Vitaly Abdrashitov and Yang Zhao and Lin Su and Xiaoqing Wang and Chujie Zheng and Sarang Metkar and Rupesh Gupta and Igor Lapchuk and David N. Racca and Madhumitha Mohan and Yanbo Li and Haojun Li and Saloni Gandhi and Xueying Lu and Chetan Bhole and Ali Hooshmand and Xin Yang and Raghavan Muthuregunathan and Jiajun Zhang and Mathew Teoh and Adam Coler and Abhinav Gupta and Xiaojing Ma and Sundara Raman Ramachandran and Morteza Ramezani and Yubo Wang and Lijuan Zhang and Richard Li and Jian Sheng and Chanh Nguyen and Yen-Chi Chen and Chuanrui Zhu and Claire Zhang and Jiahao Xu and Deepti Kulkarni and Qing Lan and Arvind Subramaniam and Ata Fatahibaarzi and Steven Shimizu and Yanning Chen and Zhipeng Wang and Ran He and Zhengze Zhou and Qingquan Song and Yun Dai and Caleb Johnson and Ping Liu and Shaghayegh Gharghabi and Gokulraj Mohanasundaram and Juan Bottaro and Santhosh Sachindran and Qi Guo and Yunxiang Ren and Chengming Jiang and Di Mo and Luke Simon and Jianqiang Shen and Jingwei Wu and Wenjing Zhang},
journal= {arXiv preprint arXiv:2602.07309},
year = {2026}
}