Algorithmic pricing with independent learners and relative experience replay
Abstract
In an infinitely repeated general-sum pricing game, independent reinforcement learners may exhibit collusive behavior without any communication, raising concerns about algorithmic collusion. To better understand the learning dynamics, we incorporate agents' relative performance (RP) among competitors using experience replay (ER) techniques. Experimental results indicate that RP considerations play a critical role in long-run outcomes. Agents that are averse to underperformance converge to the Bertrand-Nash equilibrium, while those more tolerant of underperformance tend to charge supra-competitive prices. This finding also helps mitigate the overfitting issue in independent Q-learning. Additionally, the impact of relative ER varies with the number of agents and the choice of algorithms.
Cite
@article{arxiv.2102.09139,
title = {Algorithmic pricing with independent learners and relative experience replay},
author = {Bingyan Han},
journal= {arXiv preprint arXiv:2102.09139},
year = {2025}
}
Comments
To appear in ICAIF'25. An earlier version of this paper was circulated and cited under the title "Understanding algorithmic collusion with experience replay''