We introduce a dataset of concept learning tasks that helps uncover implicit biases in large language models. Using in-context concept learning experiments, we found that language models may have a bias toward upward monotonicity in quantifiers; such bias is less apparent when the model is tested by direct prompting without concept learning components. This demonstrates that in-context concept learning can be an effective way to discover hidden biases in language models.
@article{arxiv.2510.01219,
title = {Uncovering Implicit Bias in Large Language Models with Concept Learning Dataset},
author = {Leroy Z. Wang},
journal= {arXiv preprint arXiv:2510.01219},
year = {2025}
}
Comments
Presented at EurIPS 2025 Workshop - Unifying Perspectives on Learning Biases (UPLB) https://sites.google.com/view/towards-a-unified-view