Learning to Retrieve for Job Matching
Abstract
Web-scale search systems typically tackle the scalability challenge with a two-step paradigm: retrieval and ranking. The retrieval step, also known as candidate selection, often involves extracting standardized entities, creating an inverted index, and performing term matching for retrieval. Such traditional methods require manual and time-consuming development of query models. In this paper, we discuss applying learning-to-retrieve technology to enhance LinkedIns job search and recommendation systems. In the realm of promoted jobs, the key objective is to improve the quality of applicants, thereby delivering value to recruiter customers. To achieve this, we leverage confirmed hire data to construct a graph that evaluates a seeker's qualification for a job, and utilize learned links for retrieval. Our learned model is easy to explain, debug, and adjust. On the other hand, the focus for organic jobs is to optimize seeker engagement. We accomplished this by training embeddings for personalized retrieval, fortified by a set of rules derived from the categorization of member feedback. In addition to a solution based on a conventional inverted index, we developed an on-GPU solution capable of supporting both KNN and term matching efficiently.
Cite
@article{arxiv.2402.13435,
title = {Learning to Retrieve for Job Matching},
author = {Jianqiang Shen and Yuchin Juan and Shaobo Zhang and Ping Liu and Wen Pu and Sriram Vasudevan and Qingquan Song and Fedor Borisyuk and Kay Qianqi Shen and Haichao Wei and Yunxiang Ren and Yeou S. Chiou and Sicong Kuang and Yuan Yin and Ben Zheng and Muchen Wu and Shaghayegh Gharghabi and Xiaoqing Wang and Huichao Xue and Qi Guo and Daniel Hewlett and Luke Simon and Liangjie Hong and Wenjing Zhang},
journal= {arXiv preprint arXiv:2402.13435},
year = {2024}
}