English
Related papers

Related papers: Do Offline Metrics Predict Online Performance in R…

200 papers

Procrastination, the irrational delay of tasks, is a common occurrence in online learning. Potential negative consequences include higher risk of drop-outs, increased stress, and reduced mood. Due to the rise of learning management systems…

Machine Learning · Statistics 2022-07-01 Christof Imhof , Ioan-Sorin Comsa , Martin Hlosta , Behnam Parsaeifard , Ivan Moser , Per Bergamin

We study the problems of offline and online contextual optimization with feedback information, where instead of observing the loss, we observe, after-the-fact, the optimal action an oracle with full knowledge of the objective function would…

Machine Learning · Computer Science 2023-07-04 Omar Besbes , Yuri Fonseca , Ilan Lobel

As with most Machine Learning systems, recommender systems are typically evaluated through performance metrics computed over held-out data points. However, real-world behavior is undoubtedly nuanced: ad hoc error analysis and…

Information Retrieval · Computer Science 2022-03-29 Patrick John Chia , Jacopo Tagliabue , Federico Bianchi , Chloe He , Brian Ko

Fine-tuning policies learned offline remains a major challenge in application domains. Monotonic performance improvement during \emph{fine-tuning} is often challenging, as agents typically experience performance degradation at the early…

Machine Learning · Computer Science 2025-05-05 Han Wang , Adam White , Martha White

Recommender systems continuously interact with users, creating feedback loops that shape both individual behavior and collective market dynamics. This paper introduces a simulation framework to model these loops in online retail…

Information Retrieval · Computer Science 2025-10-17 Gabriele Barlacchi , Margherita Lalli , Emanuele Ferragina , Fosca Giannotti , Luca Pappalardo

Securing long-term success is the ultimate aim of recommender systems, demanding strategies capable of foreseeing and shaping the impact of decisions on future user satisfaction. Current recommendation strategies grapple with two…

Information Retrieval · Computer Science 2025-01-14 Chongming Gao , Kexin Huang , Ziang Fei , Jiaju Chen , Jiawei Chen , Jianshan Sun , Shuchang Liu , Qingpeng Cai , Peng Jiang

The task of item recommendation requires ranking a large catalogue of items given a context. Item recommendation algorithms are evaluated using ranking metrics that depend on the positions of relevant items. To speed up the computation of…

Information Retrieval · Computer Science 2019-12-06 Steffen Rendle

In some applications of reinforcement learning, a dataset of pre-collected experience is already available but it is also possible to acquire some additional online data to help improve the quality of the policy. However, it may be…

Machine Learning · Computer Science 2023-07-11 Ruiqi Zhang , Andrea Zanette

Algorithms that aid human tasks, such as recommendation systems, are ubiquitous. They appear in everything from social media to streaming videos to online shopping. However, the feedback loop between people and algorithms is poorly…

Human-Computer Interaction · Computer Science 2022-01-19 Keith Burghardt , Kristina Lerman

Real-world intelligence systems usually operate by combining offline learning and online adaptation with highly correlated and non-stationary system data or signals, which, however, has rarely been investigated theoretically in the…

Machine Learning · Computer Science 2025-12-02 Haizheng Li , Lei Guo

A reciprocal recommendation problem is one where the goal of learning is not just to predict a user's preference towards a passive item (e.g., a book), but to recommend the targeted user on one side another user from the other side such…

Machine Learning · Computer Science 2018-06-05 Fabio Vitale , Nikos Parotsidis , Claudio Gentile

Online experiments (A/B tests) are widely regarded as the gold standard for evaluating recommender system variants and guiding launch decisions. However, a variety of biases can distort the results of the experiment and mislead…

Information Retrieval · Computer Science 2025-09-03 Chen Zheng , Zhenyu Zhao

We incorporate future information in the form of the estimated value of future gradients in online convex optimization. This is motivated by demand response in power systems, where forecasts about the current round, e.g., the weather or the…

Optimization and Control · Mathematics 2020-12-14 Antoine Lesage-Landry , Iman Shames , Joshua A. Taylor

Most modern recommendation algorithms are data-driven: they generate personalized recommendations by observing users' past behaviors. A common assumption in recommendation is that how a user interacts with a piece of content (e.g., whether…

Computers and Society · Computer Science 2024-05-12 Sarah H. Cen , Andrew Ilyas , Jennifer Allen , Hannah Li , Aleksander Madry

Bias in recommender systems not only distorts user experience but also perpetuates and amplifies existing societal stereotypes, particularly in sectors like fashion e-commerce. This study employs a dynamic modeling approach to scrutinize…

Information Retrieval · Computer Science 2025-10-28 Mahsa Goodarzi , M. Abdullah Canbaz

Modern online services continuously generate data at very fast rates. This continuous flow of data encompasses content - e.g., posts, news, products, comments -, but also user feedback - e.g., ratings, views, reads, clicks -, together with…

Information Retrieval · Computer Science 2022-01-19 João Vinagre , Alípio Mário Jorge , Marie Al-Ghossein , Albert Bifet

Synthetic data and simulators have the potential to markedly improve the performance and robustness of recommendation systems. These approaches have already had a beneficial impact in other machine-learning driven fields. We identify and…

Information Retrieval · Computer Science 2021-12-22 Adam Lesnikowski , Gabriel de Souza Pereira Moreira , Sara Rabhi , Karl Byleen-Higley

Recommendation systems are pervasive in the digital economy. An important assumption in many deployed systems is that user consumption reflects user preferences in a static sense: users consume the content they like with no other…

Computers and Society · Computer Science 2023-02-14 Andreas Haupt , Dylan Hadfield-Menell , Chara Podimata

In this work, we argue for the importance of an online evaluation budget for a reliable comparison of deep offline RL algorithms. First, we delineate that the online evaluation budget is problem-dependent, where some problems allow for less…

Machine Learning · Computer Science 2022-06-07 Vladislav Kurenkov , Sergey Kolesnikov

Deep learning-based recommendation has become a widely adopted technique in various online applications. Typically, a deployed model undergoes frequent re-training to capture users' dynamic behaviors from newly collected interaction logs.…

Information Retrieval · Computer Science 2022-04-26 Guohao Cai , Jieming Zhu , Quanyu Dai , Zhenhua Dong , Xiuqiang He , Ruiming Tang , Rui Zhang