English
Related papers

Related papers: Data Poisoning Attacks on Off-Policy Policy Evalua…

200 papers

Assessing the effects of a policy based on observational data from a different policy is a common problem across several high-stake decision-making domains, and several off-policy evaluation (OPE) techniques have been proposed. However,…

Machine Learning · Computer Science 2022-01-21 Sonali Parbhoo , Shalmali Joshi , Finale Doshi-Velez

While the classic off-policy evaluation (OPE) literature commonly assumes decision time points to be evenly spaced for simplicity, in many real-world scenarios, such as those involving user-initiated visits, decisions are made at…

Methodology · Statistics 2024-09-17 Xin Chen , Wenbin Lu , Shu Yang , Dipankar Bandyopadhyay

Off-policy evaluation (OPE) attempts to predict the performance of counterfactual policies using log data from a different policy. We extend its applicability by developing an OPE method for a class of both full support and deficient…

Machine Learning · Computer Science 2022-12-06 Yusuke Narita , Kyohei Okumura , Akihiro Shimizu , Kohei Yata

Reinforcement learning (RL) is one of the most vibrant research frontiers in machine learning and has been recently applied to solve a number of challenging problems. In this paper, we primarily focus on off-policy evaluation (OPE), one of…

Machine Learning · Statistics 2022-12-14 Masatoshi Uehara , Chengchun Shi , Nathan Kallus

We study the problem of off-policy evaluation (OPE) in Reinforcement Learning (RL), where the aim is to estimate the performance of a new policy given historical data that may have been generated by a different policy, or policies. In…

Machine Learning · Computer Science 2019-12-16 Aurélien F. Bibaut , Ivana Malenica , Nikos Vlassis , Mark J. van der Laan

Evaluating the value of a hypothetical target policy with only a logged dataset is important but challenging. On the one hand, it brings opportunities for safe policy improvement under high-stakes scenarios like clinical guidelines. On the…

Machine Learning · Computer Science 2024-10-30 Hao Sun , Alex J. Chan , Nabeel Seedat , Alihan Hüyük , Mihaela van der Schaar

Off-policy evaluation (OPE) aims to accurately evaluate the performance of counterfactual policies using only offline logged data. Although many estimators have been developed, there is no single estimator that dominates the others, because…

Machine Learning · Computer Science 2023-01-31 Takuma Udagawa , Haruka Kiyohara , Yusuke Narita , Yuta Saito , Kei Tateno

Off-policy evaluation (OPE) is to evaluate a target policy with data generated by other policies. Most previous OPE methods focus on precisely estimating the true performance of a policy. We observe that in many applications, (1) the end…

Machine Learning · Computer Science 2022-06-22 Yue Jin , Yue Zhang , Tao Qin , Xudong Zhang , Jian Yuan , Houqiang Li , Tie-Yan Liu

Reinforcement Learning aims at identifying and evaluating efficient control policies from data. In many real-world applications, the learner is not allowed to experiment and cannot gather data in an online manner (this is the case when…

Machine Learning · Computer Science 2024-07-02 Daniele Foffano , Alessio Russo , Alexandre Proutiere

In this work, we consider the problem of estimating a behaviour policy for use in Off-Policy Policy Evaluation (OPE) when the true behaviour policy is unknown. Via a series of empirical studies, we demonstrate how accurate OPE is strongly…

Machine Learning · Computer Science 2018-07-11 Aniruddh Raghu , Omer Gottesman , Yao Liu , Matthieu Komorowski , Aldo Faisal , Finale Doshi-Velez , Emma Brunskill

Off-policy evaluation (OPE) is the task of estimating the expected reward of a given policy based on offline data previously collected under different policies. Therefore, OPE is a key step in applying reinforcement learning to real-world…

Machine Learning · Computer Science 2021-03-11 Yihao Feng , Ziyang Tang , Na Zhang , Qiang Liu

Off-Policy evaluation (OPE) is concerned with evaluating a new target policy using offline data generated by a potentially different behavior policy. It is critical in a number of sequential decision making problems ranging from healthcare…

Machine Learning · Statistics 2023-01-02 Yang Xu , Chengchun Shi , Shikai Luo , Lan Wang , Rui Song

Data poisoning is a type of adversarial attack on training data where an attacker manipulates a fraction of data to degrade the performance of machine learning model. Therefore, applications that rely on external data-sources for training…

Machine Learning · Computer Science 2021-04-28 Sanjay Seetharaman , Shubham Malaviya , Rosni KV , Manish Shukla , Sachin Lodha

Off-policy policy evaluation (OPE) estimates the outcome of a new policy using historical data collected from a different policy. However, existing OPE methods cannot handle cases when the new policy introduces novel actions. This issue…

Artificial Intelligence · Computer Science 2025-02-04 Hyunji Nam , Allen Nie , Ge Gao , Vasilis Syrgkanis , Emma Brunskill

Off-policy evaluation (OPE) is a method for estimating the return of a target policy using some pre-collected observational data generated by a potentially different behavior policy. In some cases, there may be unmeasured variables that can…

Machine Learning · Statistics 2023-02-03 Yang Xu , Jin Zhu , Chengchun Shi , Shikai Luo , Rui Song

This paper addresses the problem of policy selection in domains with abundant logged data, but with a restricted interaction budget. Solving this problem would enable safe evaluation and deployment of offline reinforcement learning policies…

Off-Policy Evaluation (OPE) is employed to assess the potential impact of a hypothetical policy using logged contextual bandit feedback, which is crucial in areas such as personalized medicine and recommender systems, where online…

Machine Learning · Computer Science 2025-07-10 Yuqi Bai , Ziyu Zhao , Chenxin Lyu , Minqin Zhu , Kun Kuang

Off-policy evaluation (OPE) is the problem of evaluating new policies using historical data obtained from a different policy. In the recent OPE context, most studies have focused on single-player cases, and not on multi-player cases. In…

Machine Learning · Computer Science 2020-12-25 Kenshi Abe , Yusuke Kaneko

Evaluating a policy by deploying it in the real world can be risky and costly. Off-policy policy evaluation (OPE) algorithms use historical data collected from running a previous policy to evaluate a new policy, which provides a means for…

Artificial Intelligence · Computer Science 2017-12-07 Zhaohan Daniel Guo , Philip S. Thomas , Emma Brunskill

This paper investigates poisoning attacks against data-driven control methods. This work is motivated by recent trends showing that, in supervised learning, slightly modifying the data in a malicious manner can drastically deteriorate the…

Systems and Control · Electrical Eng. & Systems 2021-03-11 Alessio Russo , Alexandre Proutiere