English
Related papers

Related papers: Distributional Off-Policy Evaluation with Deep Qua…

200 papers

We consider evaluating and training a new policy for the evaluation data by using the historical data obtained from a different policy. The goal of off-policy evaluation (OPE) is to estimate the expected reward of a new policy over the…

Machine Learning · Statistics 2020-10-19 Masahiro Kato , Masatoshi Uehara , Shota Yasui

Offline policy evaluation (OPE) allows us to evaluate and estimate a new sequential decision-making policy's performance by leveraging historical interaction data collected from other policies. Evaluating a new policy online without a…

Machine Learning · Computer Science 2024-11-04 Allen Nie , Yash Chandak , Christina J. Yuan , Anirudhan Badrinath , Yannis Flet-Berliac , Emma Brunskil

Off-policy evaluation (OPE) attempts to predict the performance of counterfactual policies using log data from a different policy. We extend its applicability by developing an OPE method for a class of both full support and deficient…

Machine Learning · Computer Science 2022-12-06 Yusuke Narita , Kyohei Okumura , Akihiro Shimizu , Kohei Yata

Off-policy evaluation (OPE) is the problem of estimating the value of a target policy from samples obtained via different policies. Recently, applying OPE methods for bandit problems has garnered attention. For the theoretical guarantees of…

Machine Learning · Computer Science 2020-10-26 Masahiro Kato , Kenshi Abe , Kaito Ariu , Shota Yasui

Off-policy evaluation (OPE) estimates the performance of a target policy using offline data collected from a behavior policy, and is crucial in domains such as robotics or healthcare where direct interaction with the environment is costly…

In reinforcement learning, off-policy evaluation (OPE) is the problem of estimating the expected return of an evaluation policy given a fixed dataset that was collected by running one or more different policies. One of the more empirically…

Machine Learning · Computer Science 2023-10-31 Brahma S. Pavse , Josiah P. Hanna

We study distributional off-policy evaluation (OPE), of which the goal is to learn the distribution of the return for a target policy using offline data generated by a different policy. The theoretical foundation of many existing work…

Machine Learning · Statistics 2025-03-13 Sungee Hong , Zhengling Qi , Raymond K. W. Wong

Off-policy estimation (OPE) methods enable unbiased offline evaluation of recommender systems, directly estimating the online reward some target policy would have obtained, from offline data and with statistical guarantees. The theoretical…

Machine Learning · Statistics 2025-08-12 Olivier Jeunen

Evaluating a policy by deploying it in the real world can be risky and costly. Off-policy policy evaluation (OPE) algorithms use historical data collected from running a previous policy to evaluate a new policy, which provides a means for…

Artificial Intelligence · Computer Science 2017-12-07 Zhaohan Daniel Guo , Philip S. Thomas , Emma Brunskill

This study addresses the problem of off-policy evaluation (OPE) from dependent samples obtained via the bandit algorithm. The goal of OPE is to evaluate a new policy using historical data obtained from behavior policies generated by the…

Machine Learning · Statistics 2020-06-15 Masahiro Kato

Off-policy evaluation (OPE) is the task of estimating the expected reward of a given policy based on offline data previously collected under different policies. Therefore, OPE is a key step in applying reinforcement learning to real-world…

Machine Learning · Computer Science 2021-03-11 Yihao Feng , Ziyang Tang , Na Zhang , Qiang Liu

Reinforcement learning (RL) is one of the most vibrant research frontiers in machine learning and has been recently applied to solve a number of challenging problems. In this paper, we primarily focus on off-policy evaluation (OPE), one of…

Machine Learning · Statistics 2022-12-14 Masatoshi Uehara , Chengchun Shi , Nathan Kallus

We propose a robust regression approach to off-policy evaluation (OPE) for contextual bandits. We frame OPE as a covariate-shift problem and leverage modern robust regression tools. Ours is a general approach that can be used to augment any…

Machine Learning · Computer Science 2019-11-19 Anqi Liu , Hao Liu , Anima Anandkumar , Yisong Yue

Off-Policy Evaluation (OPE) is an important practical problem in algorithmic ranking systems, where the goal is to estimate the expected performance of a new ranking policy using only offline logged data collected under a different, logging…

Off-policy evaluation (OPE) is the problem of estimating the value of a target policy using historical data collected under a different logging policy. OPE methods typically assume overlap between the target and logging policy, enabling…

Methodology · Statistics 2024-03-12 Samir Khan , Martin Saveski , Johan Ugander

A key problem in off-policy Reinforcement Learning (RL) is the mismatch, or distribution shift, between the dataset and the distribution over states and actions visited by the learned policy. This problem is exacerbated in the fully offline…

Machine Learning · Computer Science 2023-11-28 Melrose Roderick , Gaurav Manek , Felix Berkenkamp , J. Zico Kolter

Matching users based on mutual preferences is a fundamental aspect of services driven by reciprocal recommendations, such as job search and dating applications. Although A/B tests remain the gold standard for evaluating new policies in…

Machine Learning · Computer Science 2025-07-21 Yudai Hayashi , Shuhei Goda , Yuta Saito

We consider the off-policy evaluation (OPE) problem in contextual bandits, where the goal is to estimate the value of a target policy using the data collected by a logging policy. Most popular approaches to the OPE are variants of the…

Machine Learning · Statistics 2024-08-20 Sutanoy Dasgupta , Yabo Niu , Kishan Panaganti , Dileep Kalathil , Debdeep Pati , Bani Mallick

Off-policy Evaluation (OPE) methods are a crucial tool for evaluating policies in high-stakes domains such as healthcare, where exploration is often infeasible, unethical, or expensive. However, the extent to which such methods can be…

Machine Learning · Computer Science 2024-04-09 Elita Lobo , Harvineet Singh , Marek Petrik , Cynthia Rudin , Himabindu Lakkaraju

In real-world recommender systems and search engines, optimizing ranking decisions to present a ranked list of relevant items is critical. Off-policy evaluation (OPE) for ranking policies is thus gaining a growing interest because it…

Machine Learning · Statistics 2022-02-04 Haruka Kiyohara , Yuta Saito , Tatsuya Matsuhiro , Yusuke Narita , Nobuyuki Shimizu , Yasuo Yamamoto