English
Related papers

Related papers: Personalization for Web-based Services using Offli…

200 papers

Offline reinforcement learning (RL) refers to the problem of learning policies from a static dataset of environment interactions. Offline RL enables extensive use and re-use of historical datasets, while also alleviating safety concerns…

Machine Learning · Computer Science 2020-12-22 Rafael Rafailov , Tianhe Yu , Aravind Rajeswaran , Chelsea Finn

In offline reinforcement learning (RL), we seek to utilize offline data to evaluate (or learn) policies in scenarios where the data are collected from a distribution that substantially differs from that of the target policy to be evaluated.…

Machine Learning · Computer Science 2021-03-09 Ruosong Wang , Yifan Wu , Ruslan Salakhutdinov , Sham M. Kakade

Offline reinforcement learning (RL) tries to learn the near-optimal policy with recorded offline experience without online exploration. Current offline RL research includes: 1) generative modeling, i.e., approximating a policy using fixed…

Machine Learning · Computer Science 2021-06-23 Hua Wei , Deheng Ye , Zhao Liu , Hao Wu , Bo Yuan , Qiang Fu , Wei Yang , Zhenhui Li

Reinforcement learning serves as a potent tool for modeling dynamic user interests within recommender systems, garnering increasing research attention of late. However, a significant drawback persists: its poor data efficiency, stemming…

Information Retrieval · Computer Science 2023-08-23 Xiaocong Chen , Siyu Wang , Julian McAuley , Dietmar Jannach , Lina Yao

How can we train a dialog model to produce better conversations by learning from human feedback, without the risk of humans teaching it harmful chat behaviors? We start by hosting models online, and gather human feedback from real-time,…

We introduce an offline reinforcement learning (RL) algorithm that explicitly clones a behavior policy to constrain value learning. In offline RL, it is often important to prevent a policy from selecting unobserved actions, since the…

Machine Learning · Computer Science 2022-06-03 Wonjoon Goo , Scott Niekum

Offline reinforcement learning aims to train a policy on a pre-recorded and fixed dataset without any additional environment interactions. There are two major challenges in this setting: (1) extrapolation error caused by approximating the…

Machine Learning · Computer Science 2023-01-31 Dmitriy Akimov , Vladislav Kurenkov , Alexander Nikulin , Denis Tarasov , Sergey Kolesnikov

The performance of reinforcement learning depends upon designing an appropriate action space, where the effect of each action is measurable, yet, granular enough to permit flexible behavior. So far, this process involved non-trivial user…

Machine Learning · Computer Science 2021-06-08 Edoardo Cetin , Oya Celiktutan

The recent development of reinforcement learning (RL) has boosted the adoption of online RL for wireless radio resource management (RRM). However, online RL algorithms require direct interactions with the environment, which may be…

Information Theory · Computer Science 2023-11-21 Kun Yang , Cong Shen , Jing Yang , Shu-ping Yeh , Jerry Sydir

Offline Reinforcement Learning (RL) is a promising approach for learning optimal policies in environments where direct exploration is expensive or unfeasible. However, the adoption of such policies in practice is often challenging, as they…

Machine Learning · Computer Science 2020-11-03 Aaron Sonabend-W , Junwei Lu , Leo A. Celi , Tianxi Cai , Peter Szolovits

Expanding reinforcement learning (RL) to offline domains generates promising prospects, particularly in sectors where data collection poses substantial challenges or risks. Pivotal to the success of transferring RL offline is mitigating…

Machine Learning · Statistics 2024-11-19 Alex Beeson , David Ireland , Giovanni Montana

Offline reinforcement learning can enable policy learning from pre-collected, sub-optimal datasets without online interactions. This makes it ideal for real-world robots and safety-critical scenarios, where collecting online data or expert…

Robotics · Computer Science 2025-08-07 Sreyas Venkataraman , Yufei Wang , Ziyu Wang , Navin Sriram Ravie , Zackory Erickson , David Held

Offline reinforcement learning provides a viable approach to obtain advanced control strategies for dynamical systems, in particular when direct interaction with the environment is not available. In this paper, we introduce a conceptual…

Machine Learning · Computer Science 2024-01-04 Marc Weber , Phillip Swazinna , Daniel Hein , Steffen Udluft , Volkmar Sterzing

Offline reinforcement learning (RL) defines a sample-efficient learning paradigm, where a policy is learned from static and previously collected datasets without additional interaction with the environment. The major obstacle to offline RL…

Machine Learning · Computer Science 2022-11-16 Yunfan Zhou , Xijun Li , Qingyu Qu

Learning a reward function from human preferences is challenging as it typically requires having a high-fidelity simulator or using expensive and potentially unsafe actual physical rollouts in the environment. However, in many tasks the…

Machine Learning · Computer Science 2023-01-05 Daniel Shin , Anca D. Dragan , Daniel S. Brown

Learning a reward function from human preferences is challenging as it typically requires having a high-fidelity simulator or using expensive and potentially unsafe actual physical rollouts in the environment. However, in many tasks the…

Machine Learning · Computer Science 2022-02-18 Daniel Shin , Daniel S. Brown , Anca D. Dragan

Offline Reinforcement Learning (RL) aims at learning an optimal control from a fixed dataset, without interactions with the system. An agent in this setting should avoid selecting actions whose consequences cannot be predicted from the…

Imitation learning has proven effective for training robots to perform complex tasks from expert human demonstrations. However, it remains limited by its reliance on high-quality, task-specific data, restricting adaptability to the diverse…

Offline reinforcement learning (offline RL) considers problems where learning is performed using only previously collected samples and is helpful for the settings in which collecting new data is costly or risky. In model-based offline RL,…

Machine Learning · Computer Science 2023-03-09 Mustafa O. Karabag , Ufuk Topcu

Offline reinforcement learning (RL) algorithms can acquire effective policies by utilizing previously collected experience, without any online interaction. It is widely understood that offline RL is able to extract good policies even from…

Machine Learning · Computer Science 2022-04-13 Aviral Kumar , Joey Hong , Anikait Singh , Sergey Levine