Removing Spurious Correlation from Neural Network Interpretations
Computation and Language
2024-12-05 v1 Artificial Intelligence
Machine Learning
Applications
Methodology
Abstract
The existing algorithms for identification of neurons responsible for undesired and harmful behaviors do not consider the effects of confounders such as topic of the conversation. In this work, we show that confounders can create spurious correlations and propose a new causal mediation approach that controls the impact of the topic. In experiments with two large language models, we study the localization hypothesis and show that adjusting for the effect of conversation topic, toxicity becomes less localized.
Keywords
Cite
@article{arxiv.2412.02893,
title = {Removing Spurious Correlation from Neural Network Interpretations},
author = {Milad Fotouhi and Mohammad Taha Bahadori and Oluwaseyi Feyisetan and Payman Arabshahi and David Heckerman},
journal= {arXiv preprint arXiv:2412.02893},
year = {2024}
}