Probe-Based Interventions for Modifying Agent Behavior
Machine Learning
2022-02-01 v1 Artificial Intelligence
Abstract
Neural nets are powerful function approximators, but the behavior of a given neural net, once trained, cannot be easily modified. We wish, however, for people to be able to influence neural agents' actions despite the agents never training with humans, which we formalize as a human-assisted decision-making problem. Inspired by prior art initially developed for model explainability, we develop a method for updating representations in pre-trained neural nets according to externally-specified properties. In experiments, we show how our method may be used to improve human-agent team performance for a variety of neural networks from image classifiers to agents in multi-agent reinforcement learning settings.
Cite
@article{arxiv.2201.12938,
title = {Probe-Based Interventions for Modifying Agent Behavior},
author = {Mycal Tucker and William Kuhl and Khizer Shahid and Seth Karten and Katia Sycara and Julie Shah},
journal= {arXiv preprint arXiv:2201.12938},
year = {2022}
}