The widespread application of Large Language Models (LLMs) involves ethical risks for users and societies. A prominent ethical risk of LLMs is the generation of unfair language output that reinforces or exacerbates harm for members of disadvantaged social groups through gender biases (Weidinger et al., 2022; Bender et al., 2021; Kotek et al., 2023). Hence, the evaluation of the fairness of LLM outputs with respect to such biases is a topic of rising interest. To advance research in this field, promote discourse on suitable normative bases and evaluation methodologies, and enhance the reproducibility of related studies, we propose a novel approach to database construction. This approach enables the assessment of gender-related biases in LLM-generated language beyond merely evaluating their degree of neutralization.
@article{arxiv.2502.20898,
title = {A database to support the evaluation of gender biases in GPT-4o output},
author = {Luise Mehner and Lena Alicija Philine Fiedler and Sabine Ammon and Dorothea Kolossa},
journal= {arXiv preprint arXiv:2502.20898},
year = {2025}
}
Comments
ISCA/ITG Workshop on Diversity in Large Speech and Language Models