Fashion-AlterEval：用于改进对话式推荐系统中备选相关物品评估的数据集

在对话式推荐系统(CRS)中，用户在每个回合提供对所推荐物品的反馈，从而引导CRS朝着更好的推荐方向发展。由于需要大量数据，用户模拟器用于训练和评估。此类用户模拟器基于对单个目标物品的知识来批评当前检索到的物品。然而，基于模拟器的离线评估受限于其对单个目标物品的关注以及在大量回合中的无限耐心。为克服这些限制，我们提出Fashion-AlterEval，一个包含人类判断用于选择备选物品的新数据集，通过在常见的时尚CRS数据集上添加新的注释来实现。 Consequently, we propose two novel meta-user simulators that use the collected judgments and allow simulated users not only to express their preferences about alternative items to their original target, but also to change their mind and level of patience. In our experiments using the Shoes and Fashion IQ as the original datasets and three CRS models, we find that using the knowledge of alternatives by the simulator can have a considerable impact on the evaluation of existing CRS models, specifically that the existing single-target evaluation underestimates their effectiveness, and when simulatedusers are allowed to instead consider alternative relevant items, the system can rapidly respond to more quickly satisfy the user.