An increasing number of studies have examined the social bias of rapidly developed large language models (LLMs). Although most of these studies have focused on bias occurring in a single social attribute, research in social science has shown that social bias often occurs in the form of intersectionality -- the constitutive and contextualized perspective on bias aroused by social attributes. In this study, we construct the Japanese benchmark inter-JBBQ, designed to evaluate the intersectional bias in LLMs on the question-answering setting. Using inter-JBBQ to analyze GPT-4o and Swallow, we find that biased output varies according to its contexts even with the equal combination of social attributes.
@article{arxiv.2506.12327,
title = {Intersectional Bias in Japanese Large Language Models from a Contextualized Perspective},
author = {Hitomi Yanaka and Xinqi He and Jie Lu and Namgi Han and Sunjin Oh and Ryoma Kumon and Yuma Matsuoka and Katsuhiko Watabe and Yuko Itatsu},
journal= {arXiv preprint arXiv:2506.12327},
year = {2025}
}
Comments
Accepted to the 6th Workshop on Gender Bias in Natural Language Processing (GeBNLP2025) at ACL2025